Plataformatec Blog Plataformatec's place to talk about Ruby, Ruby on Rails and software engineering Sat, 15 Aug 2015 09:54:05 +0000 en-US hourly 1 Working with Ecto associations and embeds Wed, 12 Aug 2015 08:00:35 +0000 »]]> This blog post aims to document how to work with associations in Ecto, covering how to read, insert, update and delete associated models and embeds. At the end, we give a more complex example that uses Ecto associations to build nested forms in Phoenix.

Note this article expects basic knowledge Ecto, particularly how repositories, models and the query syntax work. You can learn more about those in Ecto docs.


Associations in Ecto are used when two different sources (tables) are linked via foreign keys.

A classic example of this setup is “Post has many comments”. First create the two tables in migrations:

create table(:posts) do
  add :title, :string
  add :body, :text

create table(:comments) do
  add :post_id, references(:posts)
  add :body, :text

Each comment contains a post_id column that by default points to a post id.

And now define the models:

defmodule MyApp.Post do
  use Ecto.Model

  schema "posts" do
    field :title
    field :body
    has_many :comments, MyApp.Comment

defmodule MyApp.Comment do
  use Ecto.Model

  schema "comments" do
    field :body
    belongs_to :post, MyApp.Post

All the schema definitions like field, has_many and others are defined in Ecto.Schema.

Similar to has_many/3, a model can also invoke has_one/3 when the parent has at most one child entry. For example, you could think of a metadata model where “Post has one metadata” and the “Metadata belongs to post”.

The difference between has_one/3 and belongs_to/3 is that the foreign key is always defined in the model that invokes belongs_to/3. You can think of the model that calls has_* as the parent model and the one that invokes belongs_to as the child one.

This has implications specially when inserting and deleting data from the database. Because Ecto promotes database references by default, the child models (the one that calls belongs_to) must be inserted after the parent is inserted and deleted before the parent is deleted.

Querying associations

One of the benefits of defining associations is that they can be used in queries. For example:

Repo.all from p in Post,
            preload: [:comments]

Now all posts will be fetched from the database with their associated comments. The example above will perform two queries: one for loading all posts and another for loading all comments. This is often the most efficient way of loading associations from the database (even if two queries are performed) because we need to receive and parse only POSTS + COMMENTS results.

It is also possible to preload associations using joins while performing more complex queries. For example, imagine both posts and comments have votes and you want only comments with more votes than the post itself:

Repo.all from p in Post,
            join: c in assoc(p, :comments),
            where: c.votes > p.votes
            preload: [comments: c]

The example above will now perform a single query, finding all posts and the respective comments that match the criteria. Because this query performs a JOIN, the number of results returned by the database is POSTS * COMMENTS, which Ecto then processes and associates all comments into the appropriate post.

Finally, Ecto also allows data to be preloaded into models after they have been loaded via the Repo.preload/3 function:

Repo.preload posts, :comments

This is specially handy because Ecto does not support lazy loading. If you invoke post.comments and comments have not been preloaded, it will return Ecto.Association.NotLoaded. Lazy loading is often a source of confusion and performance issues and Ecto pushes developers to do the proper thing. Therefore Repo.preload/3 allow associations to be explicitly loaded anywhere, at any time.

Manipulating associations

As mentioned above, Ecto takes a more strict role in the parent-children relationship of associations. For example, if you want to create a post with comments, you can do:

Repo.transaction fn ->
  post = Repo.insert!(%Post{title: "Hello", body: "world"})

  # Build a comment from the post model
  comment =, :comments, body: "Excellent!")


The call above builds the comment using the id currently set in the post model. It is equivalent to:

%Comment{post_id:, body: "Excellent!"}

The function is specially useful in Phoenix controllers. For example, when creating the post, one would do:, :post)

As we likely want to associate the blog post to the user currently signed in the application. In another controller, we could build a comment for an existing post with:, :comments)

Ecto does not provide functions like post.comments << comment that allows mixing persisted data with non-persisted data. The only mechanism for changing both post and comments at the same time is via changesets which we will explore when talking about embeds and nested associations.

Deleting associations

When defining a has_many/3, has_one/3 and friends, you can also pass a :on_delete option that specifies which action should be performed on the associated models when the parent is deleted.

has_many :comments, MyApp.Comment, on_delete: :fetch_and_delete

Besides the value above, :delete_all and :nilify_all are also supported, with :nothing being the default. Check has_many/3 docs for more information.


Besides associations, Ecto also supports embeds in some databases. With embeds, the child is embedded inside the parent, instead of being stored in another table.

Databases like Mongo have native support for embeds. Databases like PostgreSQL uses a mixture of JSONB (embeds_one/3) and ARRAY columns to provide this functionality (both JSONB and ARRAY are supported by default and first-class citizens in Ecto).

Working with embeds is mostly the same as working with another field in a model, except when it comes to manipulating them. Let’s see an example:

defmodule MyApp.Permalink do
  use Ecto.Model

  embedded_schema do
    field :url

defmodule MyApp.Post do
  use Ecto.Model

  schema "posts" do
    field :title
    field :body
    has_many :comments, MyApp.Comment
    embeds_many :permalinks, MyApp.Permalink

Because Ecto needs to be able to track how the permalink changes inside the post, permalinks can only be added or removed via the changeset API. Let’s create a post with permalinks:

# Generate a changeset for the post
changeset = Ecto.Changeset.change(post)

# Let's track the new permalinks
changeset = Ecto.Changeset.put_change(changeset, :permalinks,
  [%Permalink{url: ""},
   %Permalink{url: ""}]

# Now let's insert the post with permalinks at once!
post = Repo.insert!(changeset)

Now if you want to replace or remove a particular permalink, you can work with permalinks as a collection and then just put it as a change again:

# Remove all permalinks from
permalinks = Enum.reject post.permalinks, fn permalink ->
  permalink.url =~ ""

# Let's create a new changeset
changeset =
  |> Ecto.Changeset.change
  |> Ecto.Changeset.put_change(:permalinks, permalinks)

# And update the entry
post = Repo.update!(changeset)

The beauty of working with changesets is that they keep track of all changes that will be sent to the database and we can introspect them at any time. For example, if we called before Repo.update!/3:


We would see something like:

[%Ecto.Changeset{action: :delete, changes: %{},
                 model: %Permalink{url: ""}},
 %Ecto.Changeset{action: :update, changes: %{},
                 model: %Permalink{url: ""}}]

If, by any chance, we were also inserting a permalink in this operation, we would see another changeset there, with action :insert.

Changesets contain a complete view of what is changing, how they are changing and you can manipulate them directly.

Nested associations and embeds

The same way we have used changesets to manipulate embeds, we can also use them to change child associations at the same time we are manipulating the parent.

One of the benefits of this feature is that we can use them to build nested forms in a Phoenix application. While nested forms in other languages and frameworks can be confusing and complex, Ecto uses changesets and explicit validations to provide a straightforward and simple way to manipulate multiple models at once.

To finish this post, let’s see an example of how to use what we have seen so far to work with nested associations in Phoenix.

Note: you will need phoenix_ecto 1.1 in order to follow this example.

First, create a new Phoenix application if you haven’t yet. The Phoenix guides can help you get started with that if it is your first time using Phoenix.

The example we will build is a classic to do list, where a list has many items. Let’s generate the TodoList resource:

mix phoenix.gen.html TodoList todo_lists title

Follow the steps printed by the command above and after let’s generate the TodoItem model:

mix phoenix.gen.model TodoItem todo_items body:text todo_list_id:references:todo_lists

Open up the MyApp.TodoList model at “web/models/todo_list.ex” and add the has_many definition inside the schema block:

has_many :todo_items, MyApp.TodoItem

Also add “todo_items” to the list of required fields in the TodoList model:

@required_fields ~w(title todo_items)

Because we have added todo_items as a required field, we are ready to submit them through the form. So let’s change our template to submit todo items too. Open up “web/templates/todo_list/form.html.eex” and add the following between the title input and the submit button:

<%= inputs_for f, :todo_items, fn i -> %>
  <div class="form-group">
    <%= label i, :body, "Task ##{i.index + 1}", class: "control-label" %>
    <%= text_input i, :body, class: "form-control" %>
    <%= if message = i.errors[:body] do %>
      <span class="help-block"><%= message %></span>
    <% end %>
<% end %>

The inputs_for/4 function comes from Phoenix.HTML.Form and it allows us to generate fields for an association or an embed, emitting a new form struct (represented by the variable i in the example above) for us to work with. Inside the inputs_for/4 function, we generate a text input for each item.

Now that we have changed the template, the final step is to change the new action in the controller to include two empty todo items by default in the todo list:

changeset = TodoList.changeset(%TodoList{todo_items: [%MyApp.TodoItem{}, %MyApp.TodoItem{}]})

Head to “http://localhost:4000/todo_lists” and you can now create a todo list with both items! However, if you try to edit the newly created todo list, you should get an error:

attempting to cast or change association :todo_items for MyApp.TodoList that was not loaded.
Please preload your associations before casting or changing the model.

As the error message says we need to preload the todo items
for both edit and update actions in MyApp.TodoListController.
Open up your controller and change the following line on both actions:

todo_list = Repo.get!(TodoList, id)


todo_list = Repo.get!(TodoList, id) |> Repo.preload(:todo_items)

Now it should also be possible to update the todo items alongside the todo list.

Both insert and update operations are ultimately powered by changesets, as we can see in our controller actions:

changeset = TodoList.changeset(todo_list, todo_list_params)

All the benefits we have discussed regarding changesets in the previous section still apply here. By inspecting the changeset before calling Repo.insert or Repo.update, it is possible to see a snapshot of all the changes that are going to happen in the database.

Not only that, the validation process behind changesets is explicit. Since we added todo_items as a required field in the todo list model, every time we call MyApp.TodoList.changeset/2, MyApp.TodoItem.changeset/2 will be called for every todo item sent through the form. The changesets returned for each todo item is then stored in the main todo list changeset (it is effectively a tree of changes).

To help us build our intuition regarding changesets a bit more, let’s add some validations to todo items and also allow them to be deleted.

Deleting todo items

Open up the MyApp.TodoItem at “web/models/todo_item.ex” and add a virtual field named :delete to the schema:

field :delete, :boolean, virtual: true

As we know the MyApp.TodoItem.changeset/2 function is the one invoked by default when manipulating todo items through todo lists. So let’s change it to the following:

@required_fields ~w(body)
@optional_fields ~w(delete) # 1. Make delete an optional field

def changeset(model, params \\ :empty) do
  |> cast(params, @required_fields, @optional_fields)
  |> validate_length(:body, min: 3)
  |> mark_for_deletion()

defp mark_for_deletion(changeset) do
  # If delete was set and it is true, let's change the action
  if get_change(changeset, :delete) do
    %{changeset | action: :delete}

We have added a call to validate_length as well as a private function that checks if the :delete field changed and, if so, we mark the changeset action to be :delete.

The functions cast, validate_length, get_change and more are all part of the Ecto.Changeset module, which is automatically imported into Ecto models.

Let’s now change our view to include the delete field. Add the following somewhere inside the inputs_for/4 call in “web/templates/todo_list/form.html.eex”:

<%= if do %>
  <span class="pull-right">
    <%= label i, :delete, "Delete?", class: "control-label" %>
    <%= checkbox i, :delete %>
<% end %>

And that’s all. Our todo items should now validate the body as well as allow deletion on update pages!

Notice we had control over the changeset and validations at all times. There are no special fields for deletion or implicit validation. Still, we were able to wire everything up with very few lines of codes.

And while the default is to call MyApp.TodoItem.changeset/2, it is possible to customize the function to be invoked when casting todo items from the todo list model. All you need to do is change the association definition:

# Call MyApp.TodoItem.nested_changeset/2 on casts
has_many :todo_items, MyApp.TodoItem, on_cast: :nested_changeset

Therefore if a model has different validation rules depending if it is sent as part of a nested association or when managed directly, we can easily keep those business rules apart by providing two different changeset functions. And because we just use functions, all the way down, they are easy to compose and test.

Summing up

In this blog post we have learned the foundations for working with associations and embeds, up to a more complex example using nested associations. If you want to further customize their behavior, read the docs for declaring the associations/embeds in Ecto.Schema or how to further manipulate changesets via Ecto.Changeset.

When it comes to the view, you can find more information on the Phoenix.HTML project, specially under the Phoenix.HTML.Form, where the inputs_for/4 function is defined.

Subscribe to Elixir Radar

]]> 5
Elixir in production interview: Adam Kittelson Thu, 06 Aug 2015 20:06:07 +0000 »]]> A few months ago we had the opportunity to interview Adam Kittelson about his experience using Elixir in production.

Adam is a senior software engineer at Brightcove. Brightcove is a global provider of cloud solutions for delivering and monetizing video across connected devices. They’re using Elixir and Phoenix to process event streams.

Watch the video below or read the interview to get to know his experiences with Elixir.

José Valim: Hi, I’m here with Adam Kittelson and we are at Erlang Factory. Adam, where do you work?

Adam Kittelson: I work at Brightcove, we do a lot of things with online video. I got started there by way of an acquisition. We were from a company called Zencoder, which they purchased three years ago, and we do online video transcoding as a cloud service.

José Valim: Cool, and I’ve heard that you are already using Elixir in production there. Can you tell us a little bit more about that?

Adam Kittelson: Yeah, a couple months ago – maybe four, five months ago – we started working on an application that allows our customers to subscribe to events. So, someone logs into our website and wants to make a change to some metadata on one of their videos, that generates a video change event which is already used by various services within the organisation, but we wanted to add the capability for the customer to receive a callback at an API endpoint that they control. So, we set up a Phoenix app that essentially attaches to the firehose of that entire event stream and just looks to see if the customer has a subscription to that event, and if they do, then we can send a message to their endpoint.

José Valim: That’s a very good use case for Elixir because sometimes you need to send tons of notification at the same time. So, if you need, you just open a bunch of connections to different clients and that would be completely ok, it’s going to definitely handle that load.

Adam Kittelson: Yeah, it’s no problem because subscriptions change very rarely we’re able to use ETS to cache those. We actually have a small cluster that is load balanced across but we use Phoenix PubSub to be able to communicate when to add things to the cache or to bust the cache. So we only have one cache miss per account essentially.

José Valim: Cool. That’s really really cool. Do you have anything to say about running Elixir in production, what was the main trigger to use Elixir?

Adam Kittelson: The main trigger to use Elixir was kind of unrelated to work. I saw it on Hacker News a year ago, in December, and I had been working on a Mud as a Rails app using websockets. It turns out websockets in Rails isn’t super awesome. And I started switching that over to Elixir and it has been working really really well.

José Valim: Awesome! Thanks a lot.

Subscribe to our blog

]]> 0
Elixir in production interview: Mickaël Rémond Wed, 05 Aug 2015 21:30:29 +0000 »]]> A few months ago we had the opportunity to interview Mickaël Rémond about his experience using Elixir in production.

Mickaël is the CEO of ProcessOne. ProcessOne is the company behind ejabberd, the famous XMPP server. They’re specialized in high-performance instant messaging solutions and they integrated Elixir into ejabberd to make ejabberd development available to a larger crowd.

Watch the video below or read the interview to get to know his experiences with Elixir.

José Valim: Hi everyone, I’m here with Mickaël, one of the creators of Ejabberd. Just a couple weeks ago they announced the beginning of the integration with Elixir. Mickaël, can you tell us a bit more about what is Ejabberd and why are you excited about Elixir?

Mickaël Rémond: Ejabberd is a large scale messaging platform. You can build a lot of different types of system with it. You can build chat systems, game platforms, machine to machine internet of things kind of platforms. When I dug deeper into Elixir I found a great opportunity to make it a language to power Ejabberd and to make it possible to write custom systems for all these kinds of applications.

Mickaël Rémond: Now that Elixir is integrated into Ejabberd, it’s very easy to write a plugin where you can extend its behaviour. For example, for the internet of things it is going to be a critical step for the platform, in my opinion.

José Valim: Great! So, you’re bringing Elixir’s productivity and expressivity with all the power and features you have in Ejabberd, and making that available to developers.

Mickaël Rémond: Yeah, exactly. It was a bit difficult to develop for Ejabberd and we’re making Ejabberd development available to a larger crowd. Our goal is to make Ejabberd a platform and build an ecosystem around real time messaging in general.

José Valim: Awesome! Thank you very much.

Mickaël Rémond: Thank you.

Subscribe to our blog

]]> 0
Elixir in production interview: Jamie Winsor Wed, 05 Aug 2015 21:26:21 +0000 »]]> A few months ago we had the opportunity to interview Jamie Winsor about his experience using Elixir in production.

Jamie is a software engineer at Undead Labs. Undead Labs is game development studio, which was founded in 2009 by Jeff Strain, a former Blizzard employee and one of the co-founders of ArenaNet. They’re using Elixir to power their distributed online game platform.

Watch the video below or read the interview to get to know his experiences with Elixir.

José Valim: Hi everyone, I’m here at Erlang Factory with Jamie Winsor. Can you tell us a little bit about where do you work, what do you do?

Jamie Winsor: I work at Undead Labs, we work on online gaming experiences. The current game we’re working on is Moonrise, it’s currently available in closed beta on PC, Mac, iOS and Android. We are creating online gaming experiences from what originally were single player games. The previous game was State of Decay on Xbox Live arcade.

My job specifically is building platform services. An online game is typically three parts. There is a game client, game server and platform. Platform is game agnostic and highly available and it’s hugely distributed. You can have any number of players connected and any number of games connected to exactly the same platform, at the same time. If you played a Blizzard game there is, if you used League of Legends there is At Undead Labs we created our own platform and it’s all built on Elixir.

José Valim: Oh, awesome! So, you already have Elixir in production?

Jamie Winsor: Oh yeah, absolutely.

José Valim: Cool. And I think it’s a quite a while that you are running it?

Jamie Winsor: Yeah! So, we started using Elixir in development since v0.9.0, maybe in August 2013. We were early adopters of Ecto as well. We adopted that really early on. It’s been great to see the community and the software grow up with our platform and it seems like it was at the right time too.

Jamie Winsor: With Elixir, we have this language that is really approachable, that is built on top the Erlang VM and it’s also extremely extensible. It has a package manager and great tooling. And specifically why I’m excited about this is that I tried to bring Erlang to my last number of companies and it was really hard to gain adoption and get people excited about it. At Undead, we now have eight engineers that work with Elixir. It was much easier to advocate. They are from the C and C++ background, never have done that functional programming, never had done even similar syntax like Ruby or Python. And they use this language (Elixir) and they’re really productive, they understood the process model.

Jamie Winsor: With Elixir you have the tooling, the syntax is actually very approachable and very familiar, the documentation is absolutely amazing and the community is super helpful.

José Valim: Thanks a lot.

Subscribe to our blog

]]> 0
Elixir in production interview: Bruce Tate Wed, 05 Aug 2015 21:24:09 +0000 »]]> A few months ago we had the opportunity to interview Bruce Tate about his experience using Elixir in production.

Bruce Tate is the CTO of icanmakeitbetter. icanmakeitbetter is a market research company. They’re using Elixir to power their chat service.

Watch the video below or read the interview to get to know his experiences with Elixir.

José Valim: Hi everyone, I’m here with Bruce Tate at Erlang Factory. Bruce, tell us a little bit more about your work, what do you do.

Bruce Tate: I’m Bruce Tate and I work at a market research firm called icanmakeitbetter. We do surveys, journals, and a new service called Chat, to help people learn more about their businesses.

José Valim: Awesome. I’ve heard you get to run Elixir in production.

Bruce Tate: We’re very excited to finally be running Elixir in production. We started working with our chat service in the Ruby language, but then we thought that we can’t be as efficient or as reliable as we needed to be with the new set of customers that we were bringing on. Elixir is going to give us the ability to do that.

José Valim: What makes you most excited about Elixir?

Bruce Tate: The scale that we can hit is absolutely fantastic, but to get to that scale we don’t have to compromise on the programmer productivity. We’re very small shop and we really need to be able to grow quickly and respond quickly to our customers.

José Valim: Thank you a lot. I hope you’re going to have tons of fun and scalability with Elixir.

Bruce Tate: Absolutely.

Subscribe to our blog

]]> 0
Coding can make you a better project manager Wed, 22 Jul 2015 12:00:46 +0000 »]]> As a Scrum Master, Project Manager or any other project management role you can have, removing impediments is part of your daily routine. They come in many forms and sizes, ranging from organizational to human ones.

But there are also technical impediments, those you annoy ask your beloved developer teammates for help if you don’t usually code.

The truth is that big part of those roadblocks can be tackled with a single line of code. Yes, you can work on low hanging fruits and have a lot of fun!

Which technical impediments can you work on?

Recently I had the opportunity to code a little bit to help the team. I can suggest the following topics, where you can achieve positive outcomes with very little effort:

Outdated project readme files

It’s just text, but it’s a good opportunity to hone your GitHub and markdown skills without breaking stuff.

Besides helping a new member on his onboarding process, it will also spread the message that it’s important to keep it up to date.

Outdated Boxen manifests

Some of our folks use Boxen to set up their environments. We often forget to update the project’s manifest. One more chance to practice. Go for it, your teammates will be grateful when they perform another clean OSX install!

Front-end fixes (color, text, positioning, size, UI behavior)

They’re small, they’re easy to fix and may bore a developer to death. But to you it’s all new and rewarding. Those represent the majority of the commits I’ve done.

As you become more experienced and confident you can expand into other areas, it’s just a matter of curiosity and opportunity.

What are the benefits?

Between one commit and another I noticed some advantages. Below are some that might also work for you:

Codebase knowledge

As you perform fixes, you get in touch with different parts of the code. It will naturally increase your awareness of where things come from and where they belong. From now on when you hear about a bug you will probably track its origin with ease.

This also makes grooming sessions more productive, since you are more familiar with the effects a new feature will bring to the application, improving your negotiation superpowers.

Better communication and relationship

Good communication saves a tremendous amount of time and energy. When you deal with code, you learn the technical vocabulary. In your next conversation with the team, people won’t need to translate jargon so you can understand. The same applies in the opposite direction.

In my opinion, a manager that can code also increases the sense of fellowship, improving the overall relationship among team members.

The feeling of shipping code

My first commit made some checkboxes to be selected by default. Nothing big, I know, but when I saw the customer using it I realized that shipping code feels amazing!

This brings the sense of belonging and increases shared ownership. You should give it a try!

Working with Pull Requests

At my very first day at Plataformatec, I realized that all teams were using Pull Requests. Shortly after, I figured out why we were using this technique. But I really understood it when I first submitted one of my own. Reading code and discussions added value to me, because I could see what was going on without interrupting people. Discussions also acted as a thermometer, revealing some insights about team relationship.

Learning how to write tests

Your code will break some tests (believe me, it will). Panic? Never! Go on and learn how to write and run automated tests. It’s not that hard. Ask your friends, they will be happy to teach you.
Learning about tests will help your communication with developers and reinforce quality culture as well.

Waste reduction

Waste is a fierce enemy and every chance to beat it must be taken. Coding can give you some opportunities to do so.

Every little fix you deal with is one less interruption and context switch for a developer. Every conversation you join in a Pull Request is a shorter feedback cycle being nurtured and less time being spent in queues.

These and other factors combined can positively impact the team’s performance.

Where to start?

I recommend following these steps:

Learning git and Pull Request basics

Here in Plataformatec we use GitHub, so finding documentation about Pull Requests wasn’t difficult. If you use it as well, you can browse Github’s Help and even use our Guidelines as a reference. If you don’t, I’m sure you can find plenty of content about your version control tool with a simple Google search.

Practicing in a controlled environment

Look for a repository that you can commit without spreading panic if you mess things up. For instance, I have practiced in our Campfire bot. If you don’t find any appropriate repository, you can fork a public project on GitHub and use it as your sandbox.

In this phase my colleagues helped me a lot crafting my first Pull Request, I bet yours will do the same. Don’t be shy and ask them for help!

Looking for and working on impediments

You have just developed a new skill, now it’s time to use it! Look for things similar to those examples I suggested and invest a little time in them. A couple of fixes later you will be able to work more independently, and the results will start to come.

Is there any pitfalls in this practice?

If you do things in a careful and balanced way, they will happen just right. But there are some traps you should avoid:

Creating dependencies

Remember that you are coding to help removing obstacles. If you turn yourself into the person responsible for maintenance, you have just become a bottleneck for the system (or an irresponsible developer, as you prefer).

Forcing people to code

Despite the several advantages mentioned, not every manager likes to code. Don’t force people into doing it.


Joining Pull Request discussions brings great value. Micromanaging doesn’t. Don’t even think of using Pull Request discussions as a control tool. Trust your team!

I hope this blog post can help you and your team to improve. Have you ever tried something like this? What are you thoughts about it? Share them with us in the comments below!

Subscribe to our blog
]]> 3