{"id":5493,"date":"2016-07-06T17:02:33","date_gmt":"2016-07-06T20:02:33","guid":{"rendered":"http:\/\/blog.plataformatec.com.br\/?p=5493"},"modified":"2017-01-31T08:22:31","modified_gmt":"2017-01-31T10:22:31","slug":"understanding-deps-and-applications-in-your-mixfile","status":"publish","type":"post","link":"http:\/\/blog.plataformatec.com.br\/2016\/07\/understanding-deps-and-applications-in-your-mixfile\/","title":{"rendered":"Understanding deps and applications in your Mixfile"},"content":{"rendered":"

Note<\/strong>: Elixir v1.4 has been released and improves on many points touched by this article. From v1.4, Elixir will automatically infer the list of applications based on your dependencies. For more information, read the official announcement<\/a>.<\/p>\n

In my journey as a curious Elixir developer, I’ve come across this seemingly simple question a few times: which applications<\/code> from third-party libraries do I need to declare in my mix.exs<\/code> file?<\/p>\n

Before we get down to the nitty-gritty of application dependencies, let’s first recap some basics of the initialization process in Elixir OTP applications.<\/p>\n

In its essence, an OTP application is a reusable software component, consisting of multiple modules of its own and it can also depend on third-party code. These dependencies can be either library applications<\/em> \u2014 a collection of standalone modules and functions, with no processes involved \u2014 or active applications<\/em>, with their own life cycles and supervision trees. This distinction is quite subtle, it took me a while to fully grasp its implications.<\/p>\n

Applications are defined with an application resource file<\/em>, like my_app.app<\/code>, which is a metadata file comprised of a single Erlang term. It includes all the information needed to start our application and is managed by Mix. If you want to dig deeper into that topic you can take a look at its documentation<\/a>. Once we create a new application with mix new my_app<\/code> a brand new mix.exs<\/code> file is generated. This is the file that customizes how my_app.app<\/code> will be assembled by Mix and it is the file we’ll be dealing with in the Elixir world. Here is what it looks like:<\/p>\n

defmodule MyApp.Mixfile do\n  use Mix.Project\n\n  def project do\n    [app: :my_app,\n     version: \"0.1.0\",\n     elixir: \"~> 1.3\",\n     build_embedded: Mix.env == :prod,\n     start_permanent: Mix.env == :prod,\n     deps: deps()]\n  end\n\n  # Configuration for the OTP application\n  #\n  # Type \"mix help compile.app\" for more information\n  def application do\n    [applications: [:logger]]\n  end\n\n  # Dependencies can be Hex packages:\n  #\n  #   {:mydep, \"~> 0.3.0\"}\n  #\n  # Or git\/path repositories:\n  #\n  #   {:mydep, git: \"https:\/\/github.com\/elixir-lang\/mydep.git\", tag: \"0.1.0\"}\n  #\n  # Type \"mix help deps\" for more examples and options\n  defp deps do\n    []\n  end\nend\n<\/code><\/pre>\n
As we can see, there are some important pieces of information in it, such as the app name, version, Elixir version requirements and so on. In addition, the application<\/code> function lets us explain what is required to boot our application: which other applications need to be started before ours, locally registered processes, which module represents the starting point of our application and also some default values for the application environment. By running mix help compile.app<\/code> we can get more information about that function. The docs<\/a>  available for the Application<\/code> behavior \u2014 which abstracts the initialization process per se<\/em> \u2014 are pretty extensive and helpful as well.<\/p>\n
So far, so good. Now things start to get interesting. Let’s take a look at the mix.exs<\/code> file of a new Phoenix project. First we generate an application skeleton with mix phoenix.new hello<\/code>, then we add {:exrm, \"~> 1.0\"}<\/code> to our deps<\/code>, including the Exrm<\/a> tool for generating our releases.<\/p>\n
defmodule Hello.Mixfile do\n  use Mix.Project\n\n  def project do\n    [app: :hello,\n     version: \"0.0.1\",\n     elixir: \"~> 1.3\",\n     # ...\n     deps: deps()]\n  end\n\n  # Configuration for the OTP application.\n  #\n  # Type `mix help compile.app` for more information.\n  def application do\n    [mod: {Hello, []},\n     applications: [:phoenix, :phoenix_pubsub, :phoenix_html, :cowboy, :logger, :gettext,\n                    :phoenix_ecto, :postgrex]]\n  end\n\n  # ...\n\n  defp deps do\n    [{:phoenix, \"~> 1.2.0\"},\n     {:phoenix_pubsub, \"~> 1.0\"},\n     {:phoenix_ecto, \"~> 3.0\"},\n     {:postgrex, \">= 0.0.0\"},\n     {:phoenix_html, \"~> 2.6\"},\n     {:phoenix_live_reload, \"~> 1.0\", only: :dev},\n     {:gettext, \"~> 0.11\"},\n     {:cowboy, \"~> 1.0\"},\n     {:exrm, \"~> 1.0\"}]\n  end\n\n  # ...\nend\n<\/code><\/pre>\n
Along with the :logger<\/code> application we’ve seen before in our vanilla Elixir app, now we have a few other applications we depend upon: some from Phoenix itself, an HTTP server, some i18n<\/em> utilities and a database driver too. If we pay close attention to the deps<\/code> and the applications<\/code> lists, we can see it is almost a 1 to 1 ratio. Every application<\/code> relates to its dep<\/code> counterpart, like cowboy<\/code>, phoenix_pubsub<\/code>, postgrex<\/code> and others. The exceptions are :logger<\/code>, which is pre-built as part of Elixir<\/a>, exrm<\/code> and phoenix_live_reload<\/code>.<\/p>\n
That mismatch confused me. I wondered if I wasn’t mixing them up (pun intended).<\/p>\n
Library applications must be included too<\/h3>\n
Why should library-only applications like phoenix_html<\/code> and gettext<\/code> be included in my application<\/code> function then? It’s not that they need to be booted up and start spawning processes or something. It turns out that our application<\/code> function has more to do with Releases<\/em> and runtime than with supervision trees. Yes, listing active applications<\/em> ensures they are started before our application, but including library applications<\/em> will also make sure they will be included<\/strong> in our installable release packages.<\/p>\n
To support that claim, we can see that even exrm<\/code> warns us when we forget to do so. Let’s try removing phoenix_html<\/code> and cowboy<\/code> from our applications<\/code>:<\/p>\n
defmodule Hello.Mixfile do\n  # ...\n  def application do\n    [mod: {Hello, []},\n     applications: [:phoenix, :phoenix_pubsub, :logger, :gettext, :phoenix_ecto, :postgrex]]\n  end\nend\n<\/code><\/pre>\n
Then make a new release:<\/p>\n
mix deps.get\nRunning dependency resolution\n* Getting phoenix (Hex package)\n  Checking package (https:\/\/repo.hex.pm\/tarballs\/phoenix-1.2.0.tar)\n  Using locally cached package\n...\n\nMIX_ENV=prod mix release\nBuilding release with MIX_ENV=prod.\n\nYou have dependencies (direct\/transitive) which are not in :applications!\nThe following apps should be added to :applications in mix.exs:\n\n        phoenix_html => phoenix_html is missing from hello\n        cowboy       => cowboy is missing from hello\n\nContinue anyway? Your release may not work as expected if these dependencies are required! [Yn]:\n<\/code><\/pre>\n
Both dependencies would be missing from the final package :bomb:. We certainly don’t want that.<\/p>\n
Development, test, docs and optional dependencies stay out<\/h3>\n
You might still ask, shouldn’t cowboy<\/code> and phoenix_html<\/code> be out of my control and get required automatically by phoenix<\/code>? That way I’d just need to worry about phoenix<\/code> as a single, rich dependency.<\/p>\n
Actually, no. From the Phoenix framework standpoint, none of these applications are actually required. Cowboy isn’t a strict runtime dependency; we can freely run our Phoenix app on another compatible web server of our choice. As for Phoenix HTML and even Gettext, they are not strictly required either. We can totally build our Phoenix app without any HTML output or localization at all, consider an API-only app for example. It’s all up to us.<\/p>\n
As we can see in Phoenix’s own mix.exs<\/code> file<\/a>, cowboy<\/code> is declared as optional: true<\/code> and phoenix_html<\/code> as only: :test<\/code>, gettext<\/code> likewise is only: :test<\/code>. The reason these dependencies are declared is that if we eventually need them in our<\/strong> app, they must be at least compatible with the Phoenix version currently in use.<\/p>\n
That also explains why exrm<\/code> and phoenix_live_reload<\/code> can be kept out of the applications list. We don’t want phoenix_live_reload<\/code> running in production, since there will be no live code reloading. As for exrm<\/code>, it is a tool used exclusively to package our application, our business code has no need to know anything about it.<\/p>\n
Cool. How come phoenix_live_reload<\/code> works in development though? Well, when we start our application \u2014 with mix phoenix.start<\/code>, for example \u2014 the code is available in the load path, but the phoenix_live_reload<\/code> application is not started yet; it only starts when we connect to its channel<\/a>. We can try it out by starting up our little Phoenix app and checking which applications get loaded.<\/p>\n
iex -S mix phoenix.start\nErlang\/OTP 18 [erts-7.3] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]\n\n[info] Running Hello.Endpoint with Cowboy using http:\/\/localhost:4000\nInteractive Elixir (1.3.0) - press Ctrl+C to exit (type h() ENTER for help)\niex(1)> Application.loaded_applications\n[\n  {:plug, 'A specification and conveniences for composable modules between web applications', '1.1.6'},\n  {:hex, 'hex', '0.12.1'},\n  ...\n]\n<\/code><\/pre>\n
A list of all the applications currently loaded is returned when we call Application.loaded_applications<\/code>. Now, after we make our first request by opening http:\/\/localhost:4000<\/code> then calling Application.loaded_applications<\/code> once again, we can see two new applications on that list: fs<\/code> and phoenix_live_reload<\/code>.<\/p>\n
iex(2)> Application.loaded_applications\n[\n  {:plug, 'A specification and conveniences for composable modules between web applications', '1.1.6'},\n  {:hex, 'hex', '0.12.1'},\n  ...\n  {:fs, 'VXZ FS Listener', '0.9.1'},\n  ...\n  {:phoenix_live_reload, 'Provides live-reload functionality for Phoenix',  '1.0.5'}\n  ...\n]\n<\/code><\/pre>\n
Declare your applications, even when you’re not using releases<\/h3>\n
I am deploying to Heroku and my application requires no release packaging whatsoever, should I still worry about my list of applications? Yes, definitely. Suppose we need to upgrade some dependencies, one of them used to be just a library application<\/em> but now has become an active application<\/em>, e.g. a cache library which now has to keep a reaper process to clean up stale data. Once we upgrade the dep<\/code> for that undeclared application<\/code>, a runtime bug is potentially introduced, given we have no guarantee our new dependencies’ applications will get started properly.<\/p>\n
As a library author, you should take extra care<\/h3>\n
The same goes for libraries, regardless of open sourcing them or not. Remember that runtime application dependencies are transitive. Let’s say an application A<\/code> depends on B<\/code>, which in turn depends on C<\/code>, the application responsible for ensuring C<\/code> is started is B<\/code>, unless C<\/code> is an optional dependency. In that case, the author of B<\/code> should document the pluggable nature of that optional dependency appropriately, or even provide code generators if applicable \u2014 as Phoenix does with cowboy<\/code>.<\/p>\n
Elixir tooling is our friend<\/h3>\n
One of the most noticeable benefits of Elixir is all the tools-support built around it. Not only in runtime, standing<\/a> on the shoulders<\/a> of Erlang, but primarily in development and build time. In case you overlooked the announcement of Elixir v1.3<\/a> recently released, you should definitely check it out. Among the improvements in dependency tracking are two new built-in mix tasks: app.tree<\/code> and deps.tree<\/code>. They are priceless timesavers in this routine task of keeping up with our dependencies.<\/p>\n
Summing up<\/h3>\n
So, when we are evaluating these dependency issues individually, it all boils down to a few simple criteria.<\/p>\n
For the deps<\/code> function in our Mixfile:<\/p>\n