[build_embedded: Mix.env == :prod,
 start_permanent: Mix.env == :prod]

Although those options were originally meant to be in Elixir v1.1, we have decided to bring them into v1.0.4 and do a new release. In this post, we will explain why.

Protocol consolidation

One of Elixir’s most important features are protocols. Protocols allow developers to write code that accept any data type, dispatching to the appropriate implementation of the protocol at runtime. For example:

defprotocol JSON do
  def encode(data)
end

defimpl JSON, for: List do
  def encode(list) do
    # ...
  end
end

We have written about protocols before and I recently explored on my Erlang Factory talk the foundation protocols have allowed us to build.

However, in order to play nicely with the dynamic nature of the Erlang VM where modules can be loaded at any time by the VM, as well as any protocol implementation, protocols need to check on every dispatch if a new implementation is available for any given data type.

While we would gladly pay this price in development as it gives developers flexibility, we would like to avoid such in production as deployments gives us a consolidated view of all modules in the system allowing us to skip those runtime checks. For this reason, Elixir provides a feature called protocol consolidation, that consolidates all protocols with their implementations, giving protocols a fast dispatch to use in production.

Prior to Elixir v1.0.4, protocol consolidation had to be manually invoked by calling mix compile.protocols, which would consolidate protocols into a predefined directory, and this directory had to be explicitly added to your load path when starting your project. Due to the manual nature of such commands, a lot of developers ended-up not running them in production, or were often confused when doing so.

For this reason, Elixir v1.0.4 introduces a :consolidate_protocols option to your projects which will take care of consolidating and loading all protocols before your application starts. This option is also set to true when :build_embedded is true.

Build embedded

When compiling your projects, Elixir will place all compiled artifacts into the _build directory:

_build/
  dev/
    lib/
      your_app/
        ebin/
        priv/
      dep1/
      dep2/
  test/
  prod/

Many of those applications and dependencies have artifacts in their source that are required during runtime. Such artifacts are placed in the priv directory in Elixir applications. By default, Elixir will symlink to their source directories during development.

In production, though, we could copy those contents to avoid symlink traversal, embedding all relevant files to run your application into the _build directory, without a need for their sources.

That’s what the :build_embedded option does and it defaults to true in production for new applications.

Start permanent

Elixir code is packaged into applications. For example, each entry we saw under _build/dev/lib above is a different application. When an application is started, it can be started in one of the three following modes:

• permanent – if app terminates, all other applications and the entire node are also terminated
• transient – if app terminates with :normal reason, it is reported but no other applications are terminated. If a transient application terminates abnormally, all other applications and the entire node are also terminated
• temporary – if app terminates, it is reported but no other applications are terminated

The default mode is temporary, which again, makes sense for development. For example, our test library called ExUnit, is also an application. If the application being tested crashes, we still want the ExUnit application to continue running in order to finish all tests and generate the proper reports. In this case, you definitely do not want your application to run as permanent.

However, in production, once your application crashes permanently, beyond recovery, we want the whole node to terminate, otherwise whatever you have monitoring your application at the operating system level won’t notice any change.

The :start_permanent option starts your application as :permanent and it defaults to true in production for new applications.

Summing up

Those new options have been introduced into Elixir v1.0.4 because they are very important for running Elixir in production. They bring more performance and stability to your Elixir-based systems.

There are other smaller changes in this release, like support for Erlang 17.5 and 18.0-rc1, as well as bug fixes. Check the release notes for more information and enjoy!

The post Build embedded and start permanent in Elixir 1.0.4 first appeared on Plataformatec Blog.

Pattern matching

Imagine you have a string with format foo=bar&token=value&bar=baz where you want to extract the value for the key token which may appear anywhere or not at all in the string.

Here is one solution a developer not very-acquainted with pattern matching would try:

def get_token(string) do
  parts = String.split(string, "&")
  Enum.find_value(parts, fn pair ->
    key_value = String.split(pair, "=")
    Enum.at(key_value, 0) == "token" && Enum.at(key_value, 1)
  end)
end

At first the code seems to work fine but once we go deeper we can see it makes many assumptions we have not really planned for!

For example, what happens if someone passes "foo=bar&token=some=value&bar=baz" as argument? The code will work and will return the string "some". But is that what we really want? Maybe we wanted "some=value" instead? Or maybe we wanted to reject it all together?

There are other examples where the code above would work by accident, possibly adding complexity to the codebase as other users may start to rely on such behaviour.

The most idiomatic way of writing the code above in Elixir is by using pattern matching:

def get_token(string) do
  parts = String.split(string, "&")
  Enum.find_value(parts, fn pair ->
    [key, value] = String.split(pair, "=")
    key == "token" && value
  end)
end

With pattern matching, we are asserting that String.split/2 is going to return a list with two elements. If someone passes "foo=bar&token&bar=baz", it will crash as the list will have only one element. If someone passes "token=some=value", it will crash too as it contains 3 items.

Our new code does not contain any of the accidental complexity of the previous one and it will also be faster. Any input that does not match the given pattern will lead to a crash, giving us the perfect opportunity to discuss and decide how to handle those corner cases.

Polymorphism is opt-in

Elixir provides protocols as a mechanism for polymorphism. A protocol allows developers to express they are willing to work with any data type, as long as it implements the protocols X, Y and Z.

I have previously compared Elixir protocols to alternatives in languages like Swift and Ruby. One nice aspect of Elixir protocols is that they are explicit, you need to explicitly outline and define a protocol for data structures to implement.

For example, one protocol in Elixir is the String.Chars protocol, which converts any data type to a string, if that data type can be converted to a human-readable string. The to_string function uses such protocol for conversions:

iex> to_string("hello")
"hello"
iex> to_string(1)
"1"
iex> to_string URI.parse("/")
"/"
iex> to_string %{hello: :world}
** (Protocol.UndefinedError) protocol String.Chars not implemented for %{hello: :world}

Imagine you have a function that converts underscores to dashes in a string:

def dasherize(string), do: String.replace(string, "_", "-")

Now imagine that at some point you decide to call to_string/1 before calling replace/3:

def dasherize(data), do: String.replace(to_string(data), "_", "-")

Albeit small, this is a drastic change to our code. Our dasherize function went from supporting only strings as argument to support a large number of data types. In other words, our code became less assertive and more generic.

That said, before adding protocols to our code, we should ask if we really intend to open our function to all types. Maybe we want dasherize to support only atoms and strings? If so, we should rather write:

def dasherize(data) when is_atom(data), do: dasherize(Atom.to_string(data))
def dasherize(data), do: String.replace(data, "_", "-")

However, if we are confident we want a protocol, then we should indeed use the protocol and write a test case that guarantees our function works for at least a couple types that implement such protocol. Such tests are extremely important to guarantee we don’t make a different assumption somewhere in the same function.

Note this trade-off does not only happen in protocols, but in any polymorphic API, like the Dict module. In practice, one should rather use specific dict implementations, like the Keyword and Map modules, and rely on the Dict module only when polymorphism is desired.

Map/struct access

Elixir provides maps, known as dictionaries in other languages, as a key-value data structure. Maps are created as follows:

map = %{name: "john", age: 42}

Maps allow two types of access. A strict access, that requires the field name to exist in the map, and a dynamic access, that returns nil if the field does not exist in the map:

# Strict access
iex> map.name
"john"
iex> map.address
** (KeyError) key :address not found in: %{age: 42, name: "john"}

# Dynamic access
iex> map[:name]
"john"
iex> map[:address]
nil

Both syntaxes have their use cases but we should prefer the strict syntax when possible as it helps us find bugs early on. The same applies to structs, which are named maps:

defmodule User do
  defstruct [:first_name, :last_name, :age]

  def name(user) do
    "#{user.first_name} #{user.last_name}"
  end
end

User.name %User{first_name: "John", last_name: "Doe"}
#=> "John Doe"

In the example above, we have defined a User struct and a name/1 function that receives the struct and returns its name. Since we are using user.first_name, if we accidentally pass a struct that does not contain such a field, it will crash immediately, with a nice error message!

In fact, the strict aspect of the user.first_name syntax is one of the reasons why structs do not support the dynamic syntax out of the box:

user = %User{first_name: "John", last_name: "Doe"}
user[:first_name]
** (Protocol.UndefinedError) protocol Access not implemented for %User{...}

In case you want to use the dynamic syntax, you need to derive the Access protocol for the User struct:

defmodule User do
  @derive [Access]
  defstruct [:first_name, :last_name, :age]

  def name(user) do
    "#{user.first_name} #{user.last_name}"
  end
end

However, only derive Access when you truly need to do so, as it is much better to push yourself to rely more on the strict syntax. I would even say relying on Access for structured data is an anti-pattern itself!

Wrapping up

The most interesting aspect of all examples above is that writing in the assertive style leads to faster, more concise and maintainable code. Even more, it allows us to focus on specific scenarios, postponing any complexity (incidental or accidental) to only when we need them, if we need them.

The post Writing assertive code with Elixir first appeared on Plataformatec Blog.

The language also feels extensible. For extensibility, I am using the same criteria we use for Elixir, which is the ability to implement language constructs using the language itself.

For example, in many languages the short-circuiting && operator is defined as special part of the language. In those languages, you can’t reimplement the operator using the constructs provided by the language.

In Elixir, however, you can implement the && operator as a macro:

defmacro left && right do
  quote do
    case unquote(left) do
      false -> false
      _ -> unquote(right)
    end
  end
end

In Swift, you can also implement operators and easily define the && operator with the help of the @auto_closure attribute:

func &&(lhs: LogicValue, rhs: @auto_closure () -> LogicValue) -> Bool {
    if lhs {
        if rhs() == true {
            return true
        }
    }
    return false
}

The @auto_closure attribute automatically wraps the tagged argument in a closure, allowing you to control when it is executed and therefore implement the short-circuiting property of the && operator.

However, one of the features I suspect that will actually hurt extensibility in Swift is the Extensions feature. I have compared the protocols implementation in Swift with the ones found in Elixir and Clojure on Twitter and, as developers have asked for a more detailed explanation, I am writing this blog post as result!

Extensions

The extension feature in Swift has many use cases. You can read them all in more detail in their documentation. For now, we will cover the general case and discuss the protocol case, which is the bulk of this blog post.

Following the example in Apple documentation itself:

extension Double {
    var km: Double { return self * 1_000.0 }
    var m: Double { return self }
    var cm: Double { return self / 100.0 }
    var mm: Double { return self / 1_000.0 }
    var ft: Double { return self / 3.28084 }
}

let oneInch = 25.4.mm
println("One inch is \(oneInch) meters")
// prints "One inch is 0.0254 meters"

let threeFeet = 3.ft
println("Three feet is \(threeFeet) meters")
// prints "Three feet is 0.914399970739201 meters"

In the example above, we are extending the Double type, adding our own computed properties. Those extensions are global and, if you are Ruby developer, it will remind you of monkey patching in Ruby. However, in Ruby classes are always open, and here the extension is always explicit (which I personally consider to be a benefit).

What troubles extensions is exactly the fact they are global. While I understand some extensions would be useful to define globally, they always come with the possibility of namespace pollution and name conflicts. Two libraries can define the same extensions to the Double type that behave slightly different, leading to bugs.

This has always been a hot topic in the Ruby community with Refinements being proposed in late 2010 as a solution to the problem. At this moment, it is unclear if extensions can be scoped in any way in Swift.

The case for protocols

Protocols are a fantastic feature in Swift. Per the documentation: “a protocol defines a blueprint of methods, properties, and other requirements that suit a particular task or piece of functionality”.

Let’s see their example:

protocol FullyNamed {
    var fullName: String { get }
}

struct Person: FullyNamed {
    var fullName: String
}

let john = Person(fullName: "John Appleseed")
// john.fullName is "John Appleseed"

In the example above we defined a FullyNamed protocol and implemented it while defining the Person struct. The benefit of protocols is that the compiler can now guarantee the struct complies with the definitions specified in the protocol. In case the protocol changes in the future, you will know immediately by recompiling your project.

I have been long advocating this feature for Ruby. For example, imagine you have the following Ruby code:

class Person
  attr_accessor :first, :last

  def full_name
    first + " " + last
  end
end

And you have a method somewhere that expects an object that implements full_name:

def print_full_name(obj)
  puts obj.full_name
end

At some point, you may want to print the title too:

def print_full_name(obj)
  if title = obj.title
    puts title + " " + obj.full_name
  else
    puts obj.full_name
  end
end

Your contract has now changed but there is no mechanism to notify implementations of such change. This is particularly cumbersome because sometimes such changes are done by accident, when you don’t want to actually modify the contract.

This issue has happened multiple times in Rails. Before Rails 3, there was no official contract between the controller and the model and between the view and the model. This meant that, while Rails worked fine with Active Record (Rails’ built-in model layer), every Rails release could possibly break integration with other models because the contract suddenly became larger due to changes in the implementation.

Since Rails 3, we actually define a contract for those interactions, but there is still no way to:

• guarantee an object complies with the contract (besides extensive use of tests)
• guarantee controllers and views obey the contract (besides extensive use of tests)

Similar to real-life contracts, unless you write it down and sign it, there is no guarantee both parts will actually maintain it.

The ideal solution is to be able to define multiple, tiny protocols. Someone using Swift would rather define multiple protocols for the controller and view layers:

protocol URL {
    func toParam() -> String
}

protocol FormErrors {
    var errors: Dict
}

The interesting aspect about Swift protocols is that you can define and implement protocols for any given type, at any time. The trouble though is that the implementation of the protocols are defined in the class/struct itself and, as such, they change the class/struct globally.

Protocols and Extensions

Since protocols in Swift are implemented directly in the class/struct, be it during definition or via extension, the protocol implementation ends up changing the class/struct globally. To see the issue with this, imagine that you have two different libraries relying on different JSON protocols:

protocol JSONA {
    func toJSON(precision: Integer) -> String
}

protocol JSONB {
    func toJSON(scale: Integer) -> String
}

If the protocols above have different specifications on how the precision argument must be handled, we will be able to implement only one of the two protocols above. That’s because implementing any of the protocols above means adding a toJSON(Integer) method to the class/struct and there can be only one of them per class/struct.

Furthermore, if implementing protocols means globally adding method to classes and structs, it can actually hinder the use of protocols as a whole, as the concerns to avoid name clashes and to avoid namespace pollution will speak louder than the protocol benefits.

Let’s contrast this with protocols in Elixir:

defprotocol JSONA do
  def to_json(data, precision)
end

defprotocol JSONB do
  def to_json(data, scale)
end

defimpl JSONA, for: Integer do
  def to_json(data, _precision) do
    Integer.to_string(data)
  end
end

JSONA.to_json(1, 10)
#=> 1

Elixir protocols are heavily influenced by Clojure protocols where the implementation of a protocol is tied to the protocol itself and not to the data type implementing the protocol. This means you can implement both JSONA and JSONB protocols for the same data types and they won’t clash!

Protocols in Elixir work by dispatching on the first argument of the protocol function. So when you invoke JSONA.to_json(1, 10), Elixir checks the first argument, sees it is an integer and dispatches to the appropriate implementation.

What is interesting is that we can actually emulate this functionality in Swift! In Swift we can define the same method multiple times, as long as the type signatures do not clash. So if we use static methods and extension, we can emulate the behaviour above:

// Define a class to act as protocol dispatch
class JSON {
}

// Implement it for Double
extension JSON {
    class func toJSON(double: Double) -> String {
        return String(double)
    }
}

// Someone may implement it later for Float too
extension JSON {
    class func toJSON(float: Float) -> String {
        return String(float)
    }
}

JSON.toJSON(2.3)

The example above emulates the dynamic dispatch ability found in Elixir and Clojure which guarantees no clashes in multiple implementations. After all, if someone defines a JSONB class, all the implementations would live in the JSONB class.

Since dynamic dispatch is already available, we hope protocols in Swift are improved to support local implementations instead of changing classes/structs globally.

Summing up

Swift is a very new language and in active development. The documentation so far doesn’t cover topics like exceptions, the module system and concurrency, which indicates there are many more exciting aspects to build, discuss and develop.

It is the first time I am excited to do some mobile development. Plus the Swift playground may become a fantastic way to introduce programming.

Finally, I would personally love if Swift protocols evolved to support non-global implementations. Protocols are a very extensible mechanism to define and implement contracts and it would be a pity to see their potential hindered due to the global side-effects it may cause to the codebase.

The post Comparing protocols and extensions in Swift and Elixir first appeared on Plataformatec Blog.