{"id":9593,"date":"2020-01-02T14:22:45","date_gmt":"2020-01-02T16:22:45","guid":{"rendered":"http:\/\/blog.plataformatec.com.br\/?p=9593"},"modified":"2020-01-03T15:22:24","modified_gmt":"2020-01-03T17:22:24","slug":"elixir-what-about-tests","status":"publish","type":"post","link":"http:\/\/blog.plataformatec.com.br\/2020\/01\/elixir-what-about-tests\/","title":{"rendered":"Elixir: What about tests?"},"content":{"rendered":"\n
\"\"<\/figure>\n\n\n\n

There is no arguing about how important tests are for our application.<\/p>\n\n\n\n

But from time to time, when we are dealing with it, some questions came up on a daily basis.<\/p>\n\n\n\n

A very common day-do-day case is our application relying on APIs and external libs, but one of the things we don’t want our test suite to do is to access the external world.<\/p>\n\n\n\n

This can lead us to many undesired scenarios, for example: if we have a connection problem on an API test that hits the internet, the test will fail, causing our test suite to be intermittent.<\/p>\n\n\n\n

How do we mock or stub a request?<\/h3>\n\n\n\n

Some libs can help us, I will talk about two of them.<\/p>\n\n\n\n

Bypass<\/h3>\n\n\n\n

The first one I wanna talk about is Bypass<\/a>.
\nIt is a lib that simulates an External Server, it works to stub the layer closer to the request. The idea is that you remove the external layer and start to call Bypass, so now, instead of talking to the internet, you are talking to a local server that you have control of. This way you can keep your tests protected.<\/p>\n\n\n\n

Let’s think about an example, we will build an application that needs to fetch data from an API, and we will create a LastfmClient<\/code> module that will be responsible for doing that.<\/p>\n\n\n

defmodule MyApp.Lastfm.Client do<\/span>\n  @moduledoc\"\"<\/span>\"\n  Search tracks\n\n  https:\/\/www.last.fm\/api\/show\/track.search\n  \"<\/span>\"\"<\/span>\n\n  @api_url \"https:\/\/ws.audioscrobbler.com\/2.0\/\"<\/span>\n\n  def search(term, url \\\\ @api_url) do<\/span>\n    response = Mojito.request(method: :get, url: search_url(url, term))\n\n    case<\/span> response do<\/span>\n      {:ok, %{status_code: 200<\/span>, body: body}} ->\n        {:ok, response(body)}\n      {:ok, %{status_code: 404<\/span>}} ->\n        {:not_found, \"Not found\"<\/span>}\n      {_, response} ->\n        {:error, response}\n    end\n  end\nend<\/code><\/div>Code language:<\/span> PHP<\/span> (<\/span>php<\/span>)<\/span><\/small><\/pre>\n\n\n
What this client does is fetch the tracks from our API and according to the type of the response return something. Not that different from most of the clients we wrote on a daily basis. Let’s test it now.<\/p>\n\n\n
describe \"search\/2\"<\/span> do<\/span>\n  test \"searches tracks by the term\"<\/span> do<\/span>\n    response = Client.search(\"The Kooks\"<\/span>)\n\n    assert {:ok, [\n      %{\n        \"artist\"<\/span> => \"The Kooks\"<\/span>,\n        \"name\"<\/span> => \"Seaside\"<\/span>,\n        \"url\"<\/span> => \"https:\/\/www.last.fm\/music\/The+Kooks\/_\/Seaside\"<\/span>\n      }\n    ]} = response\n  end\nend<\/code><\/div>Code language:<\/span> PHP<\/span> (<\/span>php<\/span>)<\/span><\/small><\/pre>\n\n\n
How can we fix it?<\/h3>\n\n\n\n
And that’s when Bypass<\/a> comes to our aid.<\/p>\n\n\n\n
First, you will need to setup Bypass in your tests.<\/p>\n\n\n
setup do<\/span>\n  bypass = Bypass.open()\n\n  {:ok, bypass<\/span>: bypass}\nend<\/code><\/div>Code language:<\/span> JavaScript<\/span> (<\/span>javascript<\/span>)<\/span><\/small><\/pre>\n\n\n
And in your test, you will set up which scenarios you want to test. Is it a success call? A not found? How should your code behave in each scenario? Tell Bypass that.<\/p>\n\n\n
describe \"search\/2\"<\/span> do<\/span>\n  test \"searches tracks by the term\"<\/span>, %{bypass: bypass} do<\/span>\n    Bypass.expect bypass, fn conn ->\n      Plug.Conn.resp(conn, 200<\/span>, payload())\n    end\n\n    response = Client.search(\"The Kooks\"<\/span>, \"http:\/\/localhost:#{bypass.port}\/\"<\/span>)\n\n    assert {:ok, [\n      %{\n        \"artist\"<\/span> => \"The Kooks\"<\/span>,\n        \"name\"<\/span> => \"Seaside\"<\/span>,\n        \"url\"<\/span> => \"https:\/\/www.last.fm\/music\/The+Kooks\/_\/Seaside\"<\/span>\n      }\n    ]} = response\n  end\nend\n\ndefp payload do<\/span>\n  ~s(\n    {\n      \"results\"<\/span>: {\n        \"@attr\"<\/span>: {},\n        \"opensearch:Query\"<\/span>: {\n          \"#text\"<\/span>: \"\"<\/span>,\n          \"role\"<\/span>: \"request\"<\/span>,\n          \"startPage\"<\/span>: \"1\"<\/span>\n        },\n        \"opensearch:itemsPerPage\"<\/span>: \"20\"<\/span>,\n        \"opensearch:startIndex\"<\/span>: \"0\"<\/span>,\n        \"opensearch:totalResults\"<\/span>: \"51473\"<\/span>,\n        \"trackmatches\"<\/span>: {\n          \"track\"<\/span>: [\n            {\n              \"artist\"<\/span>: \"The Kooks\"<\/span>,\n              \"image\"<\/span>: [\n                {\n                  \"#text\"<\/span>: \"https:\/\/lastfm.freetls.fastly.net\/i\/u\/34s\/2a96cbd8b46e442fc41c2b86b821562f.png\"<\/span>,\n                  \"size\"<\/span>: \"small\"<\/span>\n                },\n                {\n                  \"#text\"<\/span>: \"https:\/\/lastfm.freetls.fastly.net\/i\/u\/64s\/2a96cbd8b46e442fc41c2b86b821562f.png\"<\/span>,\n                  \"size\"<\/span>: \"medium\"<\/span>\n                },\n                {\n                  \"#text\"<\/span>: \"https:\/\/lastfm.freetls.fastly.net\/i\/u\/174s\/2a96cbd8b46e442fc41c2b86b821562f.png\"<\/span>,\n                  \"size\"<\/span>: \"large\"<\/span>\n                },\n                {\n                  \"#text\"<\/span>: \"https:\/\/lastfm.freetls.fastly.net\/i\/u\/300x300\/2a96cbd8b46e442fc41c2b86b821562f.png\"<\/span>,\n                  \"size\"<\/span>: \"extralarge\"<\/span>\n                }\n              ],\n              \"listeners\"<\/span>: \"851783\"<\/span>,\n              \"mbid\"<\/span>: \"c9b89088-01cd-4d98-a1f4-3e4a00519320\"<\/span>,\n              \"name\"<\/span>: \"Seaside\"<\/span>,\n              \"streamable\"<\/span>: \"FIXME\"<\/span>,\n              \"url\"<\/span>: \"https:\/\/www.last.fm\/music\/The+Kooks\/_\/Seaside\"<\/span>\n            }\n          ]\n        }\n      }\n    }\n  )\nend<\/code><\/div>Code language:<\/span> PHP<\/span> (<\/span>php<\/span>)<\/span><\/small><\/pre>\n\n\n
When to use Bypass?<\/h3>\n\n\n\n
When you have code that needs to make an HTTP request. You need to know how your application will behave. For instance, if the API is down, will your application stop working?<\/p>\n\n\n\n
But then comes some questions, if I have a module that depends on this Client<\/code> implementation, will I need to repeat this Bypass code every time in my tests?
\nWhy does another module need to know these implementation details if it is not dealing with the request?<\/p>\n\n\n\n
Mox<\/h3>\n\n\n\n
Mox<\/a> can help us with that. It forces you to implement explicit contracts in your application so we know what to expect.<\/p>\n\n\n\n
Going back to our example, let’s implement a module called Playlist<\/code> that will be responsible for fetching a list of songs by artist and give it a name.<\/p>\n\n\n
defmodule MyApp.Playlist do<\/span>\n  alias MyApp.Lastfm.Client\n\n  def artist(name) do<\/span>\n    {:ok, songs} = Client.search(name)\n\n    %{\n      name<\/span>: \"This is #{name}\"<\/span>,\n      songs<\/span>: songs\n    }\n  end\nend<\/code><\/div>Code language:<\/span> JavaScript<\/span> (<\/span>javascript<\/span>)<\/span><\/small><\/pre>\n\n\n
The simplest test we can write to this code would be something like:<\/p>\n\n\n
describe \"artist\/1\"<\/span> do<\/span>\n  test \"returns the songs by artist\"<\/span> do<\/span>\n    result = Playlist.artist(\"The Kooks\"<\/span>)\n\n    assert result[\"name\"<\/span>] == \"This is The Kooks\"<\/span>\n    assert Enum.any?(result[\"songs\"<\/span>], fn song ->\n      song[\"artist\"<\/span>] == \"The Kooks\"<\/span>\n    end)\n  end\nend<\/code><\/div>Code language:<\/span> JavaScript<\/span> (<\/span>javascript<\/span>)<\/span><\/small><\/pre>\n\n\n
Since the Playlist<\/code> depends on the Client<\/code>, to have an accurate test we would need to stub the request with the payload response from the Lastfm API so we can make sure the Playlist<\/code> behaves accordingly.<\/p>\n\n\n\n
Let’s see how we can implement those contracts.<\/p>\n\n\n\n
Behaviours<\/h3>\n\n\n\n
Elixir uses behaviours<\/a> as a way to define a set of functions that have to be implemented by a module. You can compare them to interfaces<\/a> in OOP.<\/p>\n\n\n\n
Let’s create a file at lib\/my_app\/music.ex<\/code> that says what our Client<\/code> expects as an argument and what it returns:<\/p>\n\n\n
defmodule MyApp.Music do<\/span>\n  @callback search(String<\/span>.t()) :: map()\nend<\/code><\/div>Code language:<\/span> JavaScript<\/span> (<\/span>javascript<\/span>)<\/span><\/small><\/pre>\n\n\n
In our config\/config.exs<\/code> file, let’s include two lines. The first one says which client we are using and the second one is the Lastfm API that we will remove from the default argument, just to keep the callback simple.<\/p>\n\n\n
config :my_app, :music, MyApp.Lastfm.Client\nconfig<\/span> :my_app, :lastfm_api, \"https:\/\/ws.audioscrobbler.com\/2.0\/\"<\/span><\/code><\/div>Code language:<\/span> JavaScript<\/span> (<\/span>javascript<\/span>)<\/span><\/small><\/pre>\n\n\n
In our config\/test.exs<\/code> file, let’s include our mock module.<\/p>\n\n\n
config<\/span> :my_app<\/span>, :music<\/span>, MyApp<\/span>.MusicMock<\/span><\/code><\/div>Code language:<\/span> CSS<\/span> (<\/span>css<\/span>)<\/span><\/small><\/pre>\n\n\n
In the test\/test_helper<\/code> file, let’s tell Mox which is the mock module that is responsible for mocking the calls to our behaviour.<\/p>\n\n\n
Mox<\/span>.defmock<\/span>(MyApp<\/span>.MusicMock<\/span>, for<\/span>: MyApp<\/span>.Music<\/span>)<\/code><\/div>Code language:<\/span> CSS<\/span> (<\/span>css<\/span>)<\/span><\/small><\/pre>\n\n\n
Let’s go back to our Playlist<\/code> module, and let’s change the way we call the Client<\/code> module, to use the config we just created.<\/p>\n\n\n
defmodule MyApp.Playlist do<\/span>\n  @music Application.get_env(:my_app, :music)\n\n  def artist(name) do<\/span>\n    {:ok, songs} = @music.search(name)\n\n    %{\n      \"name\"<\/span> => \"This is #{name}\"<\/span>,\n      \"songs\"<\/span> => songs\n    }\n  end\nend<\/code><\/div>Code language:<\/span> PHP<\/span> (<\/span>php<\/span>)<\/span><\/small><\/pre>\n\n\n
In our Client<\/code> module let’s adopt the behaviour we created, and let’s change the API url to fetch from the config we created.<\/p>\n\n\n
defmodule MyApp.Lastfm.Client do<\/span>\n  @moduledoc\"\"<\/span>\"\n  Search tracks\n\n  https:\/\/www.last.fm\/api\/show\/track.search\n  \"<\/span>\"\"<\/span>\n\n  @behaviour MyApp.Music\n\n  def search(term) do<\/span>\n    url = lastfm_api_url()\n    response = Mojito.request(method: :get, url: search_url(url, term))\n\n    case<\/span> response do<\/span>\n      {:ok, %{status_code: 200<\/span>, body: body}} ->\n        {:ok, response(body)}\n      {:ok, %{status_code: 404<\/span>}} ->\n        {:not_found, \"Not found\"<\/span>}\n      {_, response} ->\n        {:error, response}\n    end\n  end\n\n  defp lastfm_api_url do<\/span>\n    Application.get_env(:my_app, :lastfm_api)\n  end\nend<\/code><\/div>Code language:<\/span> PHP<\/span> (<\/span>php<\/span>)<\/span><\/small><\/pre>\n\n\n
We will need to change the test\/my_app\/lastfm\/client_test.exs<\/code> to change the env config for the API url on the setup of the test, but I’ll leave it to you to do that.<\/p>\n\n\n\n
Finally, in our PlaylistTest<\/code> we will need to import Mox<\/code>.<\/p>\n\n\n
import Mox\n\n# Make sure mocks are verified when the test exits<\/span>\nsetup :verify_on_exit!<\/code><\/div>Code language:<\/span> PHP<\/span> (<\/span>php<\/span>)<\/span><\/small><\/pre>\n\n\n
And in our test, we need to tell our MusicMock<\/code> what is expected to return.<\/p>\n\n\n
describe \"artist\/1\"<\/span> do<\/span>\n  test \"returns the songs by artist\"<\/span> do<\/span>\n    MusicMock\n    |> expect(:search, fn _name ->\n      {\n        :ok,\n        [\n          %{\n            \"artist\"<\/span> => \"The Kooks\"<\/span>,\n            \"name\"<\/span> => \"Seaside\"<\/span>,\n            \"url\"<\/span> => \"https:\/\/www.last.fm\/music\/The+Kooks\/_\/Seaside\"<\/span>\n          }\n        ]\n      }\n    end)\n\n    result = Playlist.artist(\"The Kooks\"<\/span>)\n\n    assert result[\"name\"<\/span>] == \"This is The Kooks\"<\/span>\n    assert Enum.any?(result[\"songs\"<\/span>], fn song ->\n      song[\"artist\"<\/span>] == \"The Kooks\"<\/span>\n    end)\n  end\nend<\/code><\/div>Code language:<\/span> PHP<\/span> (<\/span>php<\/span>)<\/span><\/small><\/pre>\n\n\n
What’s the difference?<\/h3>\n\n\n\n
Looking at the code it seems that we still need to pass the list of music in the tests. But there is a difference, the music’s list is a dependency of the Playlist<\/code> module, but we don’t know its internals, we don’t know from where we are fetching it, how the request response works, and all these details. The only thing we need to know at the Playlist<\/code> module is that it depends on a list of songs.<\/p>\n\n\n\n
One last thing before we go<\/h3>\n\n\n\n
We went through all that trouble to make sure the tests are protected from the outside world, but you know, Elixir has this amazing Doctest<\/code> feature, and one can argue that this replaces the application tests.<\/p>\n\n\n\n
That’s not the case, Doctests<\/code> are not tests<\/code> and you shouldn’t rely on it to make sure your application behaves the way you expect it to. Make sure you don’t use any code that hits the external world when you are writing your documentation, there is no way to mock calls and this can cause all the problems we already discussed.<\/p>\n\n\n\n
That’s all folks<\/h3>\n\n\n\n
The code from this example can be found here<\/a>, I hope it helps!<\/p>\n","protected":false},"excerpt":{"rendered":"
There is no arguing about how important tests are for our application. But from time to time, when we are dealing with it, some questions came up on a daily basis. A very common day-do-day case is our application relying on APIs and external libs, but one of the things we don’t want our test … \u00bb<\/a><\/p>\n","protected":false},"author":67,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"ngg_post_thumbnail":0,"footnotes":""},"categories":[1],"tags":[143,96],"aioseo_notices":[],"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"","_links":{"self":[{"href":"http:\/\/blog.plataformatec.com.br\/wp-json\/wp\/v2\/posts\/9593"}],"collection":[{"href":"http:\/\/blog.plataformatec.com.br\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/blog.plataformatec.com.br\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/blog.plataformatec.com.br\/wp-json\/wp\/v2\/users\/67"}],"replies":[{"embeddable":true,"href":"http:\/\/blog.plataformatec.com.br\/wp-json\/wp\/v2\/comments?post=9593"}],"version-history":[{"count":28,"href":"http:\/\/blog.plataformatec.com.br\/wp-json\/wp\/v2\/posts\/9593\/revisions"}],"predecessor-version":[{"id":9629,"href":"http:\/\/blog.plataformatec.com.br\/wp-json\/wp\/v2\/posts\/9593\/revisions\/9629"}],"wp:attachment":[{"href":"http:\/\/blog.plataformatec.com.br\/wp-json\/wp\/v2\/media?parent=9593"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/blog.plataformatec.com.br\/wp-json\/wp\/v2\/categories?post=9593"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/blog.plataformatec.com.br\/wp-json\/wp\/v2\/tags?post=9593"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
You don’t need to stub all the Client<\/code> requests in the Playlist<\/code> tests, you need to know what it returns and handle the responses, you need to have a explicit contract<\/a>.<\/p>\n\n\n\n
What’s the problem with this test?<\/h3>\n\n\n\n
It seems pretty straight forward we exercise the SUT<\/a> and verify the expected outcome, but this is a fragile test because it is accessing the external world.<\/p>\n\n\n\n
Every time we call the Client.search\/2<\/code> we are hitting the internet. A lot of problems can happen here: if the internet is down the test will fail, if the internet is slow your suit test will be slow, you won’t have the feedback as fast as you need and will be less inclined to run the test suit, or your suit test will become intermittent and you won’t trust your tests anymore, meaning that when a real failure happens you won’t care.<\/p>\n\n\n\n