Playing with Elixir and Go concurrency models

In Go Concurrency Patterns talk, Google I/O 2012, presenter Rob Pike demos some great concurrency features from Go, like channels and Go routines, and how it can be used to build fast, replicated and robust software.

Concurrency patterns is a very interesting topic but there was one statement in special that got me thinking:

“The models are equivalent but express things differently.”

This is about the distinction between Go channels and its cousin Erlang messages passing approach to communicating between processes.

Implementing Go channels with Elixir processes and messages

Before discovering that it has been done before, I wrote some simple Elixir code to play with the theoretical equivalence between those two models.

Notice that this implementation is not meant to be complete, nor efficient and is not recommended for production software. Don’t do it at home.

defmodule GoChannel do
  def make do
    spawn(&GoChannel.loop/0)
  end

  def write(channel, val) do
    send(channel, { :write, val})
  end

  def read(channel) do
    send(channel, { :read, self })

    receive do
      { :read, channel, val} -> val
    end
  end

  def loop do
    receive do
      { :read, caller } -> receive do
        { :write, val } -> send(caller, { :read, self, val }); loop
      end
    end
  end
end

Some tests

defmodule GoChannelTest do
  use ExUnit.Case

  test "write and read to a channel" do
    channel = GoChannel.make
    GoChannel.write(channel, 'hello')
    assert GoChannel.read(channel) == 'hello'
  end

  test "write and read preserves order" do
    channel = GoChannel.make
    GoChannel.write(channel, 'hello')
    GoChannel.write(channel, 'world')
    assert GoChannel.read(channel) == 'hello'
    assert GoChannel.read(channel) == 'world'
  end
end

This pseudo channel implementation relies on a combination of messages between processes to simulate the original FIFO behaviour of channels.

The same way one could pass a channel as parameter to other functions, since it’s a first-class citizen, we could pass the result of GoChannel.make, since it’s a PID, which in turn is a first-class citizen in Elixir.

Back to concurrency patterns

The first pattern demonstrated in Rob’s talk was fanIn, where two channels were combined into a single one.

func fanIn(input1, input2 
We could translate this code to Elixir, using our borrowed abstraction:
defmodule Patterns do
  def fan_in(chan1, chan2) do
    c = GoChannel.make

    spawn(loop(fn -> GoChannel.write(c, GoChannel.read(chan1)) end))
    spawn(loop(fn -> GoChannel.write(c, GoChannel.read(chan2)) end))

    c
  end

  defp loop(task) do
    fn -> task.(); loop(task) end
  end
end

Some tests:
defmodule PatternsTest do
  use ExUnit.Case

  test "fan_in combines two channels into a single one" do
    chan1 = GoChannel.make
    chan2 = GoChannel.make

    c = Patterns.fan_in(chan1, chan2)

    GoChannel.write(chan1, 'hello')
    GoChannel.write(chan2, 'world')

    assert GoChannel.read(c) == 'hello'
    assert GoChannel.read(c) == 'world'
  end
end

We could go even further in this mimic game and try to implement the select statement, but that would be a very extensive one. First let’s reflect a little about composing more complex functionality with channels.
Channels as Streams
From a consumers perspective, reading from a channel is like getting values out of a stream. So, one could wrap a channel in a stream, using the Stream.unfold/2 function:
  def stream(channel) do
    Stream.unfold(channel,
                  fn channel -> {read(channel), channel} end)
  end

This function returns a Stream, which gives us lots of power to compose using its module functions like map/2, zip/2, filter/2, and so on.
One test to demo that:
  test "compose channel values with streams" do
    channel = GoChannel.make
    stream = GoChannel.stream(channel)

    GoChannel.write(channel, 1)
    GoChannel.write(channel, 2)
    GoChannel.write(channel, 3)

    doubles = Stream.map(stream, &(&1 * 2)) |> Stream.take(2) |> Enum.to_list

    assert doubles == [2, 4]
  end

Reviewing comparisons
The following quote from Rob Pike's talk is one common analogy used to compare channels and Erlang concurrency models:

  “Rough analogy: writing to a file by name (process, Erlang) vs. writing to a file descriptor (channel, Go).”

I think analogies are really useful for communication but I believe they work better as the start of an explanation, not its summarization. So I think we could detail differences a little further.
For example, PIDs are not like “file names” since they are anonymous and automatically generated. As we just saw, PID is a first-class citizen and in the language’s perspective, is just as flexible as a channel.
I would say that channels abstraction reinforce isolation from producer to consumer, in the sense that Go routines writing to a channel doesn't know when nor who is going to consume that information. But it doesn't mean that using processes and messages one could not achieve the same level of isolation, as we just demoed.
On the other hand, identifying producers and consumers explicitly allow us to monitor and supervise them, allowing language like Erlang and Elixir to leverage the so-called supervision trees useful for building fault-tolerant software in those languages.
Besides been an interesting exercise to mimic Go’s way of solving problems, one should be aware that Erlang and Elixir have their own abstractions and patterns for handling concurrency.

For example, one could use the GenEvent module to implement a pub/sub functionality.
Elixir, Erlang and Go have some common goals, like the ones cited in the first paragraph of this post, but they also have their specifics. Embracing differences provides better results in the long term because it helps leverage each language power.
References

Tags: elixir, Posted in English, Comments Off