In the last decade, Plataformatec has also created and maintained a range of open source projects. In the next paragraphs, I will describe what the next steps are for the open source projects currently owned by Plataformatec.

We are very proud of the contributions we have made to open source communities throughout our history. We have built important projects in the Rails community, such as Devise, Simple Form, among others. We have also developed the Elixir programming language, which started as a Research & Development project inside Plataformatec, and today is used by companies around the world.

So what will happen with these projects from now on?

All of the projects above have one thing in common: they were created and maintained by passionate individuals who wanted to make positive contributions to their communities. Without these individuals and their efforts, these projects would not have become what they are today. Therefore, it is only fair that Plataformatec gives these individuals control of these projects moving forward.

For the Elixir programming language in particular, José Valim and the Elixir Core Team will continue developing and maintaining the programming language in the same capacity as they have been doing over the last few years, independently from Plataformatec and Nubank. We are in touch with the Elixir Core Team to transfer all assets, including the existing trademarks and the Elixir website, to their control. The remaining Elixir projects will be transferred to José Valim and his team at Dashbit.

For all other Open Source projects under the Plataformatec organization on GitHub, we will reach out to the current maintainers and take the necessary steps to transfer the ownership of all relevant assets, including names, source code, and logos of such projects.

We are thankful to these communities for all the support and feedback they’ve given us throughout the years. We are glad to have been the home to these projects and to have worked with the teams that will continue shaping these projects from now on.

The post Important information about our Elixir and Ruby Open Source projects first appeared on Plataformatec Blog.

There is no arguing about how important tests are for our application.

But from time to time, when we are dealing with it, some questions came up on a daily basis.

A very common day-do-day case is our application relying on APIs and external libs, but one of the things we don’t want our test suite to do is to access the external world.

This can lead us to many undesired scenarios, for example: if we have a connection problem on an API test that hits the internet, the test will fail, causing our test suite to be intermittent.

How do we mock or stub a request?

Some libs can help us, I will talk about two of them.

Bypass

The first one I wanna talk about is Bypass.
It is a lib that simulates an External Server, it works to stub the layer closer to the request. The idea is that you remove the external layer and start to call Bypass, so now, instead of talking to the internet, you are talking to a local server that you have control of. This way you can keep your tests protected.

Let’s think about an example, we will build an application that needs to fetch data from an API, and we will create a LastfmClient module that will be responsible for doing that.

defmodule MyApp.Lastfm.Client do
  @moduledoc"""
  Search tracks

  https://www.last.fm/api/show/track.search
  """

  @api_url "https://ws.audioscrobbler.com/2.0/"

  def search(term, url \\ @api_url) do
    response = Mojito.request(method: :get, url: search_url(url, term))

    case response do
      {:ok, %{status_code: 200, body: body}} ->
        {:ok, response(body)}
      {:ok, %{status_code: 404}} ->
        {:not_found, "Not found"}
      {_, response} ->
        {:error, response}
    end
  end
end
Code language: PHP (php)

What this client does is fetch the tracks from our API and according to the type of the response return something. Not that different from most of the clients we wrote on a daily basis. Let’s test it now.

describe "search/2" do
  test "searches tracks by the term" do
    response = Client.search("The Kooks")

    assert {:ok, [
      %{
        "artist" => "The Kooks",
        "name" => "Seaside",
        "url" => "https://www.last.fm/music/The+Kooks/_/Seaside"
      }
    ]} = response
  end
end
Code language: PHP (php)

What’s the problem with this test?

It seems pretty straight forward we exercise the SUT and verify the expected outcome, but this is a fragile test because it is accessing the external world.

Every time we call the Client.search/2 we are hitting the internet. A lot of problems can happen here: if the internet is down the test will fail, if the internet is slow your suit test will be slow, you won’t have the feedback as fast as you need and will be less inclined to run the test suit, or your suit test will become intermittent and you won’t trust your tests anymore, meaning that when a real failure happens you won’t care.

How can we fix it?

And that’s when Bypass comes to our aid.

First, you will need to setup Bypass in your tests.

setup do
  bypass = Bypass.open()

  {:ok, bypass: bypass}
end
Code language: JavaScript (javascript)

And in your test, you will set up which scenarios you want to test. Is it a success call? A not found? How should your code behave in each scenario? Tell Bypass that.

describe "search/2" do
  test "searches tracks by the term", %{bypass: bypass} do
    Bypass.expect bypass, fn conn ->
      Plug.Conn.resp(conn, 200, payload())
    end

    response = Client.search("The Kooks", "http://localhost:/#{bypass.port}/")

    assert {:ok, [
      %{
        "artist" => "The Kooks",
        "name" => "Seaside",
        "url" => "https://www.last.fm/music/The+Kooks/_/Seaside"
      }
    ]} = response
  end
end

defp payload do
  ~s(
    {
      "results": {
        "@attr": {},
        "opensearch:Query": {
          "#text": "",
          "role": "request",
          "startPage": "1"
        },
        "opensearch:itemsPerPage": "20",
        "opensearch:startIndex": "0",
        "opensearch:totalResults": "51473",
        "trackmatches": {
          "track": [
            {
              "artist": "The Kooks",
              "image": [
                {
                  "#text": "https://lastfm.freetls.fastly.net/i/u/34s/2a96cbd8b46e442fc41c2b86b821562f.png",
                  "size": "small"
                },
                {
                  "#text": "https://lastfm.freetls.fastly.net/i/u/64s/2a96cbd8b46e442fc41c2b86b821562f.png",
                  "size": "medium"
                },
                {
                  "#text": "https://lastfm.freetls.fastly.net/i/u/174s/2a96cbd8b46e442fc41c2b86b821562f.png",
                  "size": "large"
                },
                {
                  "#text": "https://lastfm.freetls.fastly.net/i/u/300x300/2a96cbd8b46e442fc41c2b86b821562f.png",
                  "size": "extralarge"
                }
              ],
              "listeners": "851783",
              "mbid": "c9b89088-01cd-4d98-a1f4-3e4a00519320",
              "name": "Seaside",
              "streamable": "FIXME",
              "url": "https://www.last.fm/music/The+Kooks/_/Seaside"
            }
          ]
        }
      }
    }
  )
end
Code language: PHP (php)

When to use Bypass?

When you have code that needs to make an HTTP request. You need to know how your application will behave. For instance, if the API is down, will your application stop working?

But then comes some questions, if I have a module that depends on this Client implementation, will I need to repeat this Bypass code every time in my tests?
Why does another module need to know these implementation details if it is not dealing with the request?

Mox

Mox can help us with that. It forces you to implement explicit contracts in your application so we know what to expect.

Going back to our example, let’s implement a module called Playlist that will be responsible for fetching a list of songs by artist and give it a name.

defmodule MyApp.Playlist do
  alias MyApp.Lastfm.Client

  def artist(name) do
    {:ok, songs} = Client.search(name)

    %{
      name: "This is #{name}",
      songs: songs
    }
  end
end
Code language: JavaScript (javascript)

The simplest test we can write to this code would be something like:

describe "artist/1" do
  test "returns the songs by artist" do
    result = Playlist.artist("The Kooks")

    assert result["name"] == "This is The Kooks"
    assert Enum.any?(result["songs"], fn song ->
      song["artist"] == "The Kooks"
    end)
  end
end
Code language: JavaScript (javascript)

Since the Playlist depends on the Client, to have an accurate test we would need to stub the request with the payload response from the Lastfm API so we can make sure the Playlist behaves accordingly.

You don’t need to stub all the Client requests in the Playlist tests, you need to know what it returns and handle the responses, you need to have a explicit contract.

Let’s see how we can implement those contracts.

Behaviours

Elixir uses behaviours as a way to define a set of functions that have to be implemented by a module. You can compare them to interfaces in OOP.

Let’s create a file at lib/my_app/music.ex that says what our Client expects as an argument and what it returns:

defmodule MyApp.Music do
  @callback search(String.t()) :: map()
end
Code language: JavaScript (javascript)

In our config/config.exs file, let’s include two lines. The first one says which client we are using and the second one is the Lastfm API that we will remove from the default argument, just to keep the callback simple.

config :my_app, :music, MyApp.Lastfm.Client
config :my_app, :lastfm_api, "https://ws.audioscrobbler.com/2.0/"
Code language: JavaScript (javascript)

In our config/test.exs file, let’s include our mock module.

config :my_app, :music, MyApp.MusicMock
Code language: CSS (css)

In the test/test_helper file, let’s tell Mox which is the mock module that is responsible for mocking the calls to our behaviour.

Mox.defmock(MyApp.MusicMock, for: MyApp.Music)
Code language: CSS (css)

Let’s go back to our Playlist module, and let’s change the way we call the Client module, to use the config we just created.

defmodule MyApp.Playlist do
  @music Application.get_env(:my_app, :music)

  def artist(name) do
    {:ok, songs} = @music.search(name)

    %{
      "name" => "This is #{name}",
      "songs" => songs
    }
  end
end
Code language: PHP (php)

In our Client module let’s adopt the behaviour we created, and let’s change the API url to fetch from the config we created.

defmodule MyApp.Lastfm.Client do
  @moduledoc"""
  Search tracks

  https://www.last.fm/api/show/track.search
  """

  @behaviour MyApp.Music

  def search(term) do
    url = lastfm_api_url()
    response = Mojito.request(method: :get, url: search_url(url, term))

    case response do
      {:ok, %{status_code: 200, body: body}} ->
        {:ok, response(body)}
      {:ok, %{status_code: 404}} ->
        {:not_found, "Not found"}
      {_, response} ->
        {:error, response}
    end
  end

  defp lastfm_api_url do
    Application.get_env(:my_app, :lastfm_api)
  end
end
Code language: PHP (php)

We will need to change the test/my_app/lastfm/client_test.exs to change the env config for the API url on the setup of the test, but I’ll leave it to you to do that.

Finally, in our PlaylistTest we will need to import Mox.

import Mox

# Make sure mocks are verified when the test exits
setup :verify_on_exit!
Code language: PHP (php)

And in our test, we need to tell our MusicMock what is expected to return.

describe "artist/1" do
  test "returns the songs by artist" do
    MusicMock
    |> expect(:search, fn _name ->
      {
        :ok,
        [
          %{
            "artist" => "The Kooks",
            "name" => "Seaside",
            "url" => "https://www.last.fm/music/The+Kooks/_/Seaside"
          }
        ]
      }
    end)

    result = Playlist.artist("The Kooks")

    assert result["name"] == "This is The Kooks"
    assert Enum.any?(result["songs"], fn song ->
      song["artist"] == "The Kooks"
    end)
  end
end
Code language: PHP (php)

What’s the difference?

Looking at the code it seems that we still need to pass the list of music in the tests. But there is a difference, the music’s list is a dependency of the Playlist module, but we don’t know its internals, we don’t know from where we are fetching it, how the request response works, and all these details. The only thing we need to know at the Playlist module is that it depends on a list of songs.

One last thing before we go

We went through all that trouble to make sure the tests are protected from the outside world, but you know, Elixir has this amazing Doctest feature, and one can argue that this replaces the application tests.

That’s not the case, Doctests are not tests and you shouldn’t rely on it to make sure your application behaves the way you expect it to. Make sure you don’t use any code that hits the external world when you are writing your documentation, there is no way to mock calls and this can cause all the problems we already discussed.

That’s all folks

The code from this example can be found here, I hope it helps!

The post Elixir: What about tests? first appeared on Plataformatec Blog.

This sharing often leads to confusion. Do they provide distinct behaviors? Do they overlap? For instance, is there any purpose to Elixir’s fault tolerance if Kubernetes also provides self-healing?

In this article, I will go over many of these topics and show how they are mostly complementary and discuss the rare case where they do overlap.

Self-healing

Kubernetes automatically restarts or replaces containers that fail. It can also kill containers that don’t respond to your user-defined health check. Similarly, in Erlang and Elixir, you structure your code with the help of supervisors, which automatically restart parts of your application in case of failures.

Kubernetes provides fault-tolerance within the cluster, Erlang/Elixir provide it within your application. To understand this better, let’s take an application that has to talk to a database (or any other external system). Most languages handle this by keeping a pool of database connections.

If your database goes offline, because of a bad configuration or a hardware failure, both the database and the Erlang/Elixir systems will respond negatively to health checks, which would cause Kubernetes to act and potentially relocate them. This is a node-wide failure and Kubernetes got your back.

However, what happens when part of your connections to the database are sporadically failing? For example, imagine your system is under load and you suddenly started running into connection limits, such as MySQL’s prepared statement limit. This failure likely won’t cause any health check to fail but your code will fail whenever one of its many connections reach said limit. Can you reason about this error today in your applications? Can you confidently say that the faulty connection will be dropped? Will another connection be started in place of the faulty one? Can you comfortably say this error won’t cascade in the application bringing the remaining of the connection pool down?

Erlang/Elixir’s abstractions for fault tolerance allow you to reason about those questions at the language level. It provides a mechanism for you to reason about connections, resources, in-memory state, background workers, etc. You can explicitly say how they are started, how they are shut down, and what should happen when things go wrong. These features can also be extremely helpful in face of partial failures. For example, imagine you have a news website and the live stock ticker is down. Should the website continue running, potentially serving stale data, or should everything crash down? The mental model provided by Erlang/Elixir allows us to reason about these scenarios. And of course, you can always let failures bubble up after a few retries, or even immediately, so it becomes a node-wide failure to be handled by K8s.

In a nutshell, Kubernetes and containers provide isolation and an ability to restart individual nodes when they fail, but it is not a replacement for isolation and fault handling within your own software, regardless of your language of choice. Using K8s and Erlang/Elixir allow you to apply similar self-healing and fault-tolerance principles in the large (cluster) and in the small (language/instance).

Service discovery and Distributed Erlang

The Erlang VM also provides Distributed Erlang, which allows you to exchange messages between different instances running on the same or different machines. In Elixir, this is as easy as:

for node <- Node.list() do
  # Send :hello_world message to named process "MyProcess" in each node
  send {node, MyProcess}, :hello_world
end
Code language: PHP (php)

When running in distributed mode (which is not a requirement in any way and you need to explicitly enable it), the Erlang VM will automatically serialize and deserialize the data as well as make sure the connection between nodes is alive, but it does not provide any node discovery. It is the programmer responsibility to say exactly where each node is located and connect the nodes together.

Luckily, Kubernetes provides service discovery out of the box. This means that, K8s allows us to fully automate the node discovery, which would otherwise be manual and error prone. Libraries like libcluster do exactly that (and rolling your own wouldn’t be complicated either). This is another great example of where Kubernetes and the Erlang VM complement each other!

However, you may still be wondering, is there a benefit to running Distributed Erlang when Kubernetes’ Service Discovery makes it relatively easy to have systems communicating with each other? Especially when considering RPC protocols such as Thrift, gRPC, and others?

When we are talking about different languages and different systems communicating with each other, picking one of the existing RPC mechanisms is likely the best choice, and they will also work fine with Erlang/Elixir. The scenario where the Erlang VM really shines, in my opinion, is for building homogeneous systems, i.e. when you have multiple deployments of the same container and they exchange information. For example, imagine you are building a real-time application when you want to track which users are in the same chat room, or in the same city block, or in the same mountain track. As users connect and disconnect and as nodes are brought up and down, you could somehow update the database or communicate via a complex RPC mechanism, while carefully watching the cluster for topology changes.

With the Erlang VM, you can just broadcast or exchange this information directly, without having to worry about serialization protocols, connection management, etc, as everything is provided by the VM. All without external dependencies. This is one of the many features that makes Phoenix a breeze to build distributed web-realtime systems.

Automated rollouts vs Hot code swapping

When it comes to deployment, Kubernetes automatically rolls out changes to your application or its configuration, avoiding changing all instances at the same time. At the same time, the Erlang VM supports hot code swapping, which allows you to change the code that is running in production within a single instance without shutting said instance down.

Those two deployments techniques are obviously conflicting. In fact, hot code swapping does not go well in general with the whole idea of immutable containers. Does it mean that Kubernetes and the Erlang VM are a poor fit? Not really, because you don’t have to use hot code swapping. In fact, most people do not. Most Elixir applications are deployed using blue-green, canary, or similar techniques.

The truth about hot code swapping is that it is actually complicated to pull off in practice. Let’s use the database as an example once again. When you are deploying a new version of your software, whenever you update your database, you should never perform destructive changes. For example, if you want to rename a column, you have to add a new column, migrate the data over, and then remove the column. If you just rename the column, then you will have failures whenever doing rollouts, because you will have two versions of the software running at the same (one using the old column and the other using the new one). In hot code swapping, we have precisely the same issue, except it applies to all states inside your application. Companies that use hot code swapping often report they spend as much time developing the software as testing the upgrades themselves.

Of course, it doesn’t mean hot code swapping is useless. The Erlang VM development is mostly driven by business needs and there was a legitimate need for hot code swapping. In particular, when building telephone switches, there is never an appropriate moment to shut down an instance for updates, because at any given time a system is full of long running connections, perhaps days or even weeks. So being able to upgrade a live system is extremely helpful. If you have a similar need, then hot code swapping may be an option. Another option is to have smarter clients and migrate client connections between nodes when deploying.

Hot code swapping can also be used under other circumstances, such as during development to provide live code loading, without a need to restart your server, or to replace smaller components in production that don’t require replacing the whole instance.

Configuration management and Configuration providers

Another feature provided by both Elixir and Kubernetes is configuration management. However, as seen before, they work at very distinct levels. While Elixir provides a unified API for configuring applications, it is relatively low-level. In a production system, you often want both configuration and secrets to be managed by higher level tools, such as the ones provided by Kubernetes. Luckily, you can incorporate said configuration tools into your deployment workflow with the help of Configuration Providers. This functionality is part of Elixir releases, which were officially made part of the Elixir language in version 1.9.

Stay alert: pod resources

When provisioning Erlang and Elixir with Kubernetes, it is important to stay alert to one particular configuration: pod resources.

When using other technologies, it is common practice to break a large node into a bunch of small pods/containers. For example, if you have a node with 8 cores, you could allocate half of each CPU to a pod and split the memory equally between them, on a total of 16 pods.

This approach makes sense in many technologies that cannot exploit CPU and I/O concurrency simultaneously. However, the Erlang VM excels at managing system resources and your system will most likely be more efficient if you assign large pods to your Erlang and Elixir applications instead of breaking it apart into a bunch of small ones.

If the Erlang VM is sharing a machine with other applications you may want to consider reducing busy waiting. By doing so, the VM will optimize for lower CPU usage, making it a better neighbor, but with slightly higher latencies.

Summing up

Kubernetes and the Erlang VM work at distinct levels. Kubernetes orchestrates within a cluster, the Erlang VM orchestrates at the language level within an instance. Fred Hebert summed up this distinction well in a tweet:

Still seeing bad comparisons between kubernetes and #Erlang/OTP. K8s is to OTP what region failover is to k8s. They operate on different layers of abstraction and impact distinct components.

OTP allows handling partial failures WITHIN an instance, something k8s can’t help with.— Fred Hebert (@mononcqc) April 29, 2019

If you are using Erlang/Elixir and you wonder how Kubernetes applies compared to other languages, you can use Kubernetes for the Erlang VM as you would with any other technology. Given that Erlang/Elixir software can typically scale both horizontally and vertically, it gives you many options on how you want to allocate your resources within K8s.

On other areas, Kubernetes and the Erlang VM can nicely complement each other, such as using K8s Service Discovery to connect Erlang VM instances. Of course, Distributed Erlang is not a requirement and Erlang/Elixir are great languages even for stateless apps, thanks to its scalability and reliability.

If you are one of the few who really need hot code swapping in production, then the Erlang VM may be one of the best platforms to do so, but keep in mind you will be straying away from the common path in both technologies.

Finally, if you appreciate Kubernetes and its concepts, you may enjoy working with Erlang and Elixir, as they will give you an opportunity to apply similar idioms on the small and on the large.

Thanks to Fernando Tapia Rico, Fred Hebert, George Guimarães, Tristan Sloughter, and Wojtek Mach for reviewing this article.

The post Kubernetes and the Erlang VM: orchestration on the large and the small first appeared on Plataformatec Blog.

To give some background information, Hexdocs.pm started out as basically just static file hosting for documentation. With the introduction of private Hexdocs it became a distinct Elixir application. Over time, we have also moved handling of documentation tarballs there to offload API servers. Instead of API servers doing all the work, they now just upload a tarball to S3 which automatically sends a SQS message which is then picked up by the Hexdocs app. The initial implementation of Hexdocs pipeline was done with a custom GenStage producer and a consumer.

Updating the pipeline to use Broadway was really straightforward. We’ve completely removed our custom producer and replaced it with BroadwaySQS.Producer. In terms of consuming messages, our code is pretty much unchanged, instead of implementing GenStage.handle_events/3 callback we now implement Broadway.handle_message/3.

Previously, we needed to configure our supervision tree to start X producers and Y consumers, and set consumers to be subscribed to producers. With Broadway, we specify the desired topology and it starts all processes under a dedicated supervisor. Not only it’s a more declarative approach, Broadway automatically adds a “Terminator” process to the supervision tree that ensures proper application shutdown. While before the application could abort a job in the middle of processing, now Broadway ensures the job queue is drained before shutting down the app.

On the testing front, we didn’t start our GenStage pipeline at all during tests to avoid doing network requests, and we tested the logic through internal APIs. Now, we’re conditionally using Broadway.DummyProducer, which doesn’t hit the network, and we’re triggering an event in the pipeline using Broadway.test_messages/2 making the test more realistic.

Perhaps the biggest win by moving over to Broadway was that it automatically batches and acknowledges messages. This, along with other existing and planned future features like rate-limiting and backoff, is what is most appealing about Broadway – that the community best practices will usually be the default behaviour or just a matter of configuration.

Overall, we were very happy with updating Hexdocs to use Broadway and we’ve been running it in production for last few months without issues. Not only we removed a lot of code, we got a couple nice features for free and we will continue to reap the benefits as Broadway gets updated.

See hexpm/hexdocs#11 to see all required code changes.

The post Using Broadway at Hexdocs.pm first appeared on Plataformatec Blog.

Today we are happy to announce MiniRepo, a minimal Hex server that can be used for packages self-hosting.

MiniRepo ships with the following features:

• Pluggable storage (with built-in adapters for local filesystem and Amazon S3)
• Mirroring
• Publishing packages via HTTP API
• Support for multiple repositories and mirrors

The goal of MiniRepo is to be minimal so that it can be easy to understand and can serve as a starting point for a more complete solution that may make sense in a given organization.

See instructions for usage with Mix and Rebar3.

Finally, by making it easier to run self-hosted Hex registry we are achieving one of the goals of the Building and Packaging Working Group at Erlang Ecosystem Foundation, which we are glad to contribute to!

The post Announcing MiniRepo, a minimal Hex server first appeared on Plataformatec Blog.

Installing Elixir master

(Update: This section is no longer relevant since v1.9 is already out!)

Since Elixir v1.9 is not out yet, we need to use the development version. Locally, my preferred approach is to use the Elixir plugin for the asdf-vm version manager.

Here’s a couple of ways we may use asdf to install recent development versions:

# install latest master
$ asdf install elixir master
$ asdf local elixir master

# or, install particular revision:
$ asdf install elixir ref:b8b7e5a
$ asdf local elixir ref:b8b7e5a

Per “Deployment” section of mix release documentation:

A release is built on a host, a machine which contains Erlang, Elixir, and any other dependencies needed to compile your application. A release is then deployed to a target, potentially the same machine as the host, but usually separate, and often there are many targets (either multiple instances, or the release is deployed to heterogeneous environments).

We deploy Hex.pm using Docker containers and we needed to change our Dockerfile. If you’re deploying using buildpacks (e.g. to Heroku or Gigalixir), it should be as simple as setting elixir_version=master in your elixir_buildpack.config.

Setting up releases

Elixir 1.9 ships with two new Mix tasks to work with releases:

• mix release.init – generates sample files for releases
• mix release – builds the release

The sample files generated by mix release.init are optional, if they are not present in your project then the release will be built with default options.

On Hex.pm, previously we were building releases using Distillery and to work with Elixir releases we needed to make a few small tweaks. Here are the main ones:

• add :releases section to mix.exs – this is an optional step but since we don’t deploy on Windows, we only need to generate executable files for UNIX-like systems
• replace rel/vm.args with rel/vm.args.eex
• replace rel/hooks/pre_configure with rel/env.sh.eex
• add config/releases.exs for runtime configuration of the release
• remove Distillery dependency (remember to mix deps.unlock it!)

See the “Replace Distillery with Elixir releases” PR on Hex.pm repo for more details.

We now have a few files that deal with configuring our app/release, let’s take a step back and see what they can do:

• config/prod.exs – provides build-time application configuration
• config/releases.exs – provides runtime application configuration. We’re using the new Config module and the System.fetch_env!/1 function, also introduced in Elixir v1.9.0, to conveniently return the environment variable if set, or raise an error.
• rel/vm.args.eex – provides a static mechanism for configuring the Erlang Virtual Machine and other runtime flags. For now, we use the defaults but if down the line we’d tune the VM, we’d set the options here.
• rel/env.sh.eex – provides a dynamic mechanism for setting up the VM, runtime flags, and environment variables.

RELEASE_NODE and RELEASE_COOKIE variables are used by the release script, see “Environment variables” section in the documentation for all recognized variables. The POD_A_RECORD variable we have there is specific to our deployment environment on Hex.pm, we deploy it to Google Kubernetes Engine.

See “Application configuration” and “vm.args and env.sh (env.bat)” sections for more information.

Finally, we use the mix release task to actually assemble the release:

$ mix release
* assembling hexpm-0.0.1 on MIX_ENV=dev
* using config/releases.exs to configure the release at runtime
* creating _build/dev/rel/hexpm/releases/0.0.1/vm.args
* creating _build/dev/rel/hexpm/releases/0.0.1/env.sh

Release created at _build/dev/rel/hexpm!

# To start your system
_build/dev/rel/hexpm/bin/hexpm start

Once the release is running:

# To connect to it remotely
_build/dev/rel/hexpm/bin/hexpm remote

# To stop it gracefully (you may also send SIGINT/SIGTERM)
_build/dev/rel/hexpm/bin/hexpm stop

To list all commands:

_build/dev/rel/hexpm/bin/hexpm

Running the release

The generated release script (bin/hexpm) has many commands:

$ _build/dev/rel/hexpm/bin/hexpm
Usage: hexpm COMMAND [ARGS]

The known commands are:

start          Starts the system
start_iex      Starts the system with IEx attached
daemon         Starts the system as a daemon
daemon_iex     Starts the system as a daemon with IEx attached
eval "EXPR"    Executes the given expression on a new, non-booted system
rpc "EXPR"     Executes the given expression remotely on the running system
remote         Connects to the running system via a remote shell
restart        Restarts the running system via a remote command
stop           Stops the running system via a remote command
pid            Prints the OS PID of the running system via a remote command
version        Prints the release name and version to be booted

In our Hex.pm deployment we have used two of these commands for now:

• bin/hexpm start – we use it as the start command to be run in our Docker container
• bin/hexpm eval – we use it to run DB migrations and other maintenance scripts. For migrations, the command is: bin/hexpm eval 'Hexpm.ReleaseTasks.migrate()'.

Summary

In this blog post we’ve walked through using Elixir releases on an existing project, Hex.pm. We’ve installed the development version of Elixir, configured the release, and adjusted our deployment setup to use it. Hex.pm was previously using Distillery, and with minimal changes we were able to update it to use built-in releases support.

Overall, I’m very happy about this change. We’ve ended up with about the same amount of configuration code, but I think it’s a little bit better structured and more obvious.

I especially like new conventions around configuration. Where previously we used workarounds like config :app, {:system, "ENV_VAR"} and "${ENV_VAR}" (and REPLACE_OS_VARS=true), we now have a clear distinction between build-time and runtime configuration. mix release documentation does a really good job of explaining configuration aspects in particular but also the whole release process in general.

Building the release is now faster too, on my machine ~2.5s now vs ~5.5s before. Granted, it’s probably the least concern but it’s a nice cherry on top nonetheless.

As of this writing, Hex.pm is already deployed using Elixir releases. Now your turn – try out releases on your project! (And if something goes wrong, submit an issue!)

The post Updating Hex.pm to use Elixir releases first appeared on Plataformatec Blog.