In the last decade, Plataformatec has also created and maintained a range of open source projects. In the next paragraphs, I will describe what the next steps are for the open source projects currently owned by Plataformatec.

We are very proud of the contributions we have made to open source communities throughout our history. We have built important projects in the Rails community, such as Devise, Simple Form, among others. We have also developed the Elixir programming language, which started as a Research & Development project inside Plataformatec, and today is used by companies around the world.

So what will happen with these projects from now on?

All of the projects above have one thing in common: they were created and maintained by passionate individuals who wanted to make positive contributions to their communities. Without these individuals and their efforts, these projects would not have become what they are today. Therefore, it is only fair that Plataformatec gives these individuals control of these projects moving forward.

For the Elixir programming language in particular, José Valim and the Elixir Core Team will continue developing and maintaining the programming language in the same capacity as they have been doing over the last few years, independently from Plataformatec and Nubank. We are in touch with the Elixir Core Team to transfer all assets, including the existing trademarks and the Elixir website, to their control. The remaining Elixir projects will be transferred to José Valim and his team at Dashbit.

For all other Open Source projects under the Plataformatec organization on GitHub, we will reach out to the current maintainers and take the necessary steps to transfer the ownership of all relevant assets, including names, source code, and logos of such projects.

We are thankful to these communities for all the support and feedback they’ve given us throughout the years. We are glad to have been the home to these projects and to have worked with the teams that will continue shaping these projects from now on.

The post Important information about our Elixir and Ruby Open Source projects first appeared on Plataformatec Blog.

There is no arguing about how important tests are for our application.

But from time to time, when we are dealing with it, some questions came up on a daily basis.

A very common day-do-day case is our application relying on APIs and external libs, but one of the things we don’t want our test suite to do is to access the external world.

This can lead us to many undesired scenarios, for example: if we have a connection problem on an API test that hits the internet, the test will fail, causing our test suite to be intermittent.

How do we mock or stub a request?

Some libs can help us, I will talk about two of them.

Bypass

The first one I wanna talk about is Bypass.
It is a lib that simulates an External Server, it works to stub the layer closer to the request. The idea is that you remove the external layer and start to call Bypass, so now, instead of talking to the internet, you are talking to a local server that you have control of. This way you can keep your tests protected.

Let’s think about an example, we will build an application that needs to fetch data from an API, and we will create a LastfmClient module that will be responsible for doing that.

defmodule MyApp.Lastfm.Client do
  @moduledoc"""
  Search tracks

  https://www.last.fm/api/show/track.search
  """

  @api_url "https://ws.audioscrobbler.com/2.0/"

  def search(term, url \\ @api_url) do
    response = Mojito.request(method: :get, url: search_url(url, term))

    case response do
      {:ok, %{status_code: 200, body: body}} ->
        {:ok, response(body)}
      {:ok, %{status_code: 404}} ->
        {:not_found, "Not found"}
      {_, response} ->
        {:error, response}
    end
  end
end
Code language: PHP (php)

What this client does is fetch the tracks from our API and according to the type of the response return something. Not that different from most of the clients we wrote on a daily basis. Let’s test it now.

describe "search/2" do
  test "searches tracks by the term" do
    response = Client.search("The Kooks")

    assert {:ok, [
      %{
        "artist" => "The Kooks",
        "name" => "Seaside",
        "url" => "https://www.last.fm/music/The+Kooks/_/Seaside"
      }
    ]} = response
  end
end
Code language: PHP (php)

What’s the problem with this test?

It seems pretty straight forward we exercise the SUT and verify the expected outcome, but this is a fragile test because it is accessing the external world.

Every time we call the Client.search/2 we are hitting the internet. A lot of problems can happen here: if the internet is down the test will fail, if the internet is slow your suit test will be slow, you won’t have the feedback as fast as you need and will be less inclined to run the test suit, or your suit test will become intermittent and you won’t trust your tests anymore, meaning that when a real failure happens you won’t care.

How can we fix it?

And that’s when Bypass comes to our aid.

First, you will need to setup Bypass in your tests.

setup do
  bypass = Bypass.open()

  {:ok, bypass: bypass}
end
Code language: JavaScript (javascript)

And in your test, you will set up which scenarios you want to test. Is it a success call? A not found? How should your code behave in each scenario? Tell Bypass that.

describe "search/2" do
  test "searches tracks by the term", %{bypass: bypass} do
    Bypass.expect bypass, fn conn ->
      Plug.Conn.resp(conn, 200, payload())
    end

    response = Client.search("The Kooks", "http://localhost:/#{bypass.port}/")

    assert {:ok, [
      %{
        "artist" => "The Kooks",
        "name" => "Seaside",
        "url" => "https://www.last.fm/music/The+Kooks/_/Seaside"
      }
    ]} = response
  end
end

defp payload do
  ~s(
    {
      "results": {
        "@attr": {},
        "opensearch:Query": {
          "#text": "",
          "role": "request",
          "startPage": "1"
        },
        "opensearch:itemsPerPage": "20",
        "opensearch:startIndex": "0",
        "opensearch:totalResults": "51473",
        "trackmatches": {
          "track": [
            {
              "artist": "The Kooks",
              "image": [
                {
                  "#text": "https://lastfm.freetls.fastly.net/i/u/34s/2a96cbd8b46e442fc41c2b86b821562f.png",
                  "size": "small"
                },
                {
                  "#text": "https://lastfm.freetls.fastly.net/i/u/64s/2a96cbd8b46e442fc41c2b86b821562f.png",
                  "size": "medium"
                },
                {
                  "#text": "https://lastfm.freetls.fastly.net/i/u/174s/2a96cbd8b46e442fc41c2b86b821562f.png",
                  "size": "large"
                },
                {
                  "#text": "https://lastfm.freetls.fastly.net/i/u/300x300/2a96cbd8b46e442fc41c2b86b821562f.png",
                  "size": "extralarge"
                }
              ],
              "listeners": "851783",
              "mbid": "c9b89088-01cd-4d98-a1f4-3e4a00519320",
              "name": "Seaside",
              "streamable": "FIXME",
              "url": "https://www.last.fm/music/The+Kooks/_/Seaside"
            }
          ]
        }
      }
    }
  )
end
Code language: PHP (php)

When to use Bypass?

When you have code that needs to make an HTTP request. You need to know how your application will behave. For instance, if the API is down, will your application stop working?

But then comes some questions, if I have a module that depends on this Client implementation, will I need to repeat this Bypass code every time in my tests?
Why does another module need to know these implementation details if it is not dealing with the request?

Mox

Mox can help us with that. It forces you to implement explicit contracts in your application so we know what to expect.

Going back to our example, let’s implement a module called Playlist that will be responsible for fetching a list of songs by artist and give it a name.

defmodule MyApp.Playlist do
  alias MyApp.Lastfm.Client

  def artist(name) do
    {:ok, songs} = Client.search(name)

    %{
      name: "This is #{name}",
      songs: songs
    }
  end
end
Code language: JavaScript (javascript)

The simplest test we can write to this code would be something like:

describe "artist/1" do
  test "returns the songs by artist" do
    result = Playlist.artist("The Kooks")

    assert result["name"] == "This is The Kooks"
    assert Enum.any?(result["songs"], fn song ->
      song["artist"] == "The Kooks"
    end)
  end
end
Code language: JavaScript (javascript)

Since the Playlist depends on the Client, to have an accurate test we would need to stub the request with the payload response from the Lastfm API so we can make sure the Playlist behaves accordingly.

You don’t need to stub all the Client requests in the Playlist tests, you need to know what it returns and handle the responses, you need to have a explicit contract.

Let’s see how we can implement those contracts.

Behaviours

Elixir uses behaviours as a way to define a set of functions that have to be implemented by a module. You can compare them to interfaces in OOP.

Let’s create a file at lib/my_app/music.ex that says what our Client expects as an argument and what it returns:

defmodule MyApp.Music do
  @callback search(String.t()) :: map()
end
Code language: JavaScript (javascript)

In our config/config.exs file, let’s include two lines. The first one says which client we are using and the second one is the Lastfm API that we will remove from the default argument, just to keep the callback simple.

config :my_app, :music, MyApp.Lastfm.Client
config :my_app, :lastfm_api, "https://ws.audioscrobbler.com/2.0/"
Code language: JavaScript (javascript)

In our config/test.exs file, let’s include our mock module.

config :my_app, :music, MyApp.MusicMock
Code language: CSS (css)

In the test/test_helper file, let’s tell Mox which is the mock module that is responsible for mocking the calls to our behaviour.

Mox.defmock(MyApp.MusicMock, for: MyApp.Music)
Code language: CSS (css)

Let’s go back to our Playlist module, and let’s change the way we call the Client module, to use the config we just created.

defmodule MyApp.Playlist do
  @music Application.get_env(:my_app, :music)

  def artist(name) do
    {:ok, songs} = @music.search(name)

    %{
      "name" => "This is #{name}",
      "songs" => songs
    }
  end
end
Code language: PHP (php)

In our Client module let’s adopt the behaviour we created, and let’s change the API url to fetch from the config we created.

defmodule MyApp.Lastfm.Client do
  @moduledoc"""
  Search tracks

  https://www.last.fm/api/show/track.search
  """

  @behaviour MyApp.Music

  def search(term) do
    url = lastfm_api_url()
    response = Mojito.request(method: :get, url: search_url(url, term))

    case response do
      {:ok, %{status_code: 200, body: body}} ->
        {:ok, response(body)}
      {:ok, %{status_code: 404}} ->
        {:not_found, "Not found"}
      {_, response} ->
        {:error, response}
    end
  end

  defp lastfm_api_url do
    Application.get_env(:my_app, :lastfm_api)
  end
end
Code language: PHP (php)

We will need to change the test/my_app/lastfm/client_test.exs to change the env config for the API url on the setup of the test, but I’ll leave it to you to do that.

Finally, in our PlaylistTest we will need to import Mox.

import Mox

# Make sure mocks are verified when the test exits
setup :verify_on_exit!
Code language: PHP (php)

And in our test, we need to tell our MusicMock what is expected to return.

describe "artist/1" do
  test "returns the songs by artist" do
    MusicMock
    |> expect(:search, fn _name ->
      {
        :ok,
        [
          %{
            "artist" => "The Kooks",
            "name" => "Seaside",
            "url" => "https://www.last.fm/music/The+Kooks/_/Seaside"
          }
        ]
      }
    end)

    result = Playlist.artist("The Kooks")

    assert result["name"] == "This is The Kooks"
    assert Enum.any?(result["songs"], fn song ->
      song["artist"] == "The Kooks"
    end)
  end
end
Code language: PHP (php)

What’s the difference?

Looking at the code it seems that we still need to pass the list of music in the tests. But there is a difference, the music’s list is a dependency of the Playlist module, but we don’t know its internals, we don’t know from where we are fetching it, how the request response works, and all these details. The only thing we need to know at the Playlist module is that it depends on a list of songs.

One last thing before we go

We went through all that trouble to make sure the tests are protected from the outside world, but you know, Elixir has this amazing Doctest feature, and one can argue that this replaces the application tests.

That’s not the case, Doctests are not tests and you shouldn’t rely on it to make sure your application behaves the way you expect it to. Make sure you don’t use any code that hits the external world when you are writing your documentation, there is no way to mock calls and this can cause all the problems we already discussed.

That’s all folks

The code from this example can be found here, I hope it helps!

The post Elixir: What about tests? first appeared on Plataformatec Blog.

Assim como qualquer modelo, framework ou ferramenta, existe uma tendência natural das pessoas acreditarem que o artefato resolverá os problemas organizacionais simplesmente por estar sendo aplicado. A partir das minhas andanças como consultor, me deparei com diversas situações onde as pessoas disseram que tentaram aplicar os OKRs e não obtiveram sucesso.

No texto a seguir, compartilharei um compilado de aprendizados e dicas para quem já lida com o framework ou para quem está considerando levá-lo para o seu dia a dia.

1) Qual a relação entre OKR e pensamento estratégico?

Um grande desafio nas organizações diz respeito a como decompor sua estratégia. Tudo começa com a construção de um pensamento estratégico que, em linhas gerais, passa por três grandes etapas: (1) coleta de informações; (2) formulação de ideias; (3) planejamento das ações (essas diretrizes e mais dicas podem ser vistas em Strategic Thinking: A Step-by-step Approach to Strategy and Leadership).

É fundamental que a organização entenda o que está mudando no mercado em que ela atua, como a concorrência tem se posicionado diante de tais mudanças e como a organização está operando hoje.

Passada a etapa de entendimento do contexto, o próximo passo de um pensamento estratégico é o levantamento de desejos, hipóteses e necessidades. É importante que a organização crie uma visão futura inspiradora e que garanta a sustentabilidade do negócio diante dos desafios que ela enfrentará.

O último passo é o planejamento daquilo que será feito. A organização precisa compreender quais são as opções de ações e, com muita clareza, deverá escolher o que será feito e o que não será. Neste momento, é essencial que haja uma clareza do que é prioridade dentro de uma perspectiva de curto, médio e longo prazo. Costumo dizer que prioridade é uma palavra que não deveria ser colocada no plural porque quando tudo é prioridade, nada é prioridade.

Pois bem, então o que OKR tem a ver com o pensamento estratégico exposto acima?

Quando falamos que o OKR é a composição do que desejo alcançar (objetivo) e como vou medir o progresso (conjunto de resultados-chave), precisamos partir de uma referência, que no caso do pensamento estratégico se dá através da coleta de informações. Se eu não conheço meu contexto atual, como eu posso definir onde eu quero chegar?

Além do mais, uma visão clara é necessária para orientar o processo de criação dos OKRs. Quando a organização não consegue estabelecer seu norte, qualquer caminho (ou no caso OKR) servirá. O problema dessa condição é que ela gera desperdício de dinheiro, esforço e energia.

Por fim, os OKRs nos forçam a fazer escolhas quando precisamos pensar no que será o foco de um trimestre, por exemplo. Ao definir poucos objetivos, estamos garantindo que a organização estará voltada naquilo que é o mais importante para o horizonte de tempo determinado.

Portanto, lembre-se: sem estratégia, não faz sentido a aplicação de OKR.

2) Quais problemas você pode resolver com OKRs?

Se você deseja comunicar a estratégia da organização, melhorar a transparência, trazer senso de propósito para as equipes e conectar o trabalho das pessoas com os resultados do negócio, OKR pode ser uma boa forma de você chegar lá.

2.1 – Comunicação da estratégia

A partir do momento em que a organização tem a sua estratégia estabelecida, os OKRs que serão desenhados pelas equipes deverão se comunicar com a mesma. Em outras palavras, caso existam OKRs que não estão conectados com as diretrizes estratégicas do negócio, eles deveriam ser descartados.

2.2 Transparência

Ao acompanhar semanalmente o progresso dos resultados-chave, validar e comunicar mensalmente os resultados dos OKRs e refinar trimestralmente os avanços em um momento de inspeção e adaptação, a organização reduzirá uma natural assimetria de informação que existe nas hierarquias e nos silos departamentais. Ficou na dúvida de como aumentar a transparência? Siga o fluxo abaixo.

Ao atingirmos as cadências mencionadas na imagem acima, aumentamos a visibilidade do que está acontecendo no negócio e deixamos os problemas palpáveis para serem tratados.

2.3 Conexão do trabalho com os resultados

Ao atrelar as iniciativas (ex: projetos, evoluções de um produto) com os resultados do negócio, trazemos um senso de propósito que pode motivar e engajar as pessoas no trabalho que está sendo feito.

Como exercício, tenho proposto uma amarração onde toda iniciativa trabalhada por uma equipe (ex: projeto, história de usuário etc.) deve ter um objetivo e uma clareza do impacto no negócio, além da sua descrição e critério de término.

O exemplo abaixo é uma situação de uma equipe que, a priori, tinha recebido a “demanda” de contratar uma solução de força de vendas. Ao conectar tal demanda com o objetivo do negócio e com os resultados-chave, a equipe conseguiu ter maior entendimento da importância do trabalho que seria feito.

Ao unir a entrega com o resultado, você garantirá a necessidade das pessoas olharem os números e os dados da organização. Além disso, você evitará que a equipe invista tempo e esforço para atender uma demanda que não gera impacto nos objetivos do negócio. E em último caso, mas nem por isso menos importante, você forçará a equipe se questionar o porquê de estarem fazendo algo que não impacta diretamente nos objetivos do negócio.

3) Qual a importância das métricas na definição dos OKRs?

Indicadores de negócio são as melhores medidas para avaliar o progresso em direção aos objetivos. Se você está em busca de sair de um modelo onde há uma cobrança por entregas (output) e deseja ser cobrado por resultado (outcome), defina resultados-chave (key results) que se acoplem com os indicadores de performance (KPI) da organização.

Lembrando que a sigla OKR é composta de duas partes (objetivos e resultados-chave), um KPI que precisa ser melhorado será um excelente ponto de partida para criar um OKR e se tornará um resultado-chave para um objetivo.

Lembre-se: OKR e KPI funcionarão perfeitamente juntos dado que os KPIs ajudam a monitorar o desempenho e a identificar problemas e áreas de melhoria no modo atual de operar da organização. Já os OKRs, contribuem na resolução de problemas, na melhoria de processos e no impulsionamento de inovações.

Para exemplificar, gostaria de compartilhar um caso real de uma organização que tinha como objetivo “Expandir os talentos da área de produto” e possuía como indicadores de performance o tempo médio de contratação, o número de vagas ocupadas e a pesquisa de satisfação. Depois de uma sessão de definição, saímos com o seguinte OKR:

Objetivo: Expandir os talentos da área de produto.

Resultados-chave:

• Contratar 4 novas pessoas de desenvolvimento para aumentar a senioridade da equipe.
• Aumentar o resultado da pesquisa de satisfação de 75% (resultado no último trimestre) para 85%.
• Reduzir o tempo médio de contratação de 45 dias para 30 dias.

4) Dicas práticas

Antes de finalizar o texto gostaria, de compartilhar 6 dicas práticas para você revisar sua estratégia de adoção de OKR:

1. Tenha o apoio executivo: OKR, assim como qualquer outra mudança organizacional precisa do suporte executivo para acontecer. Acreditar que apenas movimentos locais (debaixo para cima) farão com que a transformação aconteça é começar o jogo perdendo. Portanto, é fundamental que os OKRs da empresa se desdobrem para os times.
2. Não defina muitos objetivos: independente do contexto (empresa, equipe, tribo etc.), tenha de 2 até 3 objetivos para garantir que as pessoas estejam focadas naquilo que importa durante o trimestre.
3. Tenha resultados-chave mensuráveis: se você não consegue medir, você não saberá se está alcançado o seu objetivo. Não criei resultados-chave para métricas que não existem. Se você quer ter bons OKRs, coloque um esforço para ter as métricas disponíveis.
4. Não tenha mais do que 3 resultados-chave por objetivo: se é muito difícil de medir um objetivo talvez ela possa ser repensando, reescrito ou reestruturado.
5. Deixe os resultados visíveis para todas as pessoas da organização: visibilidade é alma para um melhor alinhamento e para que os problemas que surgirem de fato sejam resolvidos.
6. Evite resultados-chave orientados por tarefas (ex: entrega de projetos): foque em resultados orientados ao negócio porque isso ajudará a organização a traduzir os benefícios de uma nova abordagem de trabalho e suportará a ideia de equipes multidisciplinares.

Conclusão

OKR é uma ótima forma de pensar de forma estratégica e fazer com que as pessoas colaborem na execução daquilo que a organização busca conquistar. O framework em si é bem simples, porém sua adoção, nem tanto.

Pensando em sistematizar a mensagem deste post, resolvi criar um Canvas que te ajudará no momento de desenvolver os OKRs na sua organização.

Basta seguir os passos sinalizados pelas setas e responder às perguntas necessárias em cada um dos quadrantes. Comece pela estratégia, defina os objetivos, crie os resultados-chave, priorize as iniciativas e defina as áreas que precisarão colaborar com aquele OKR. Cada linha do quadro representará um OKR.

E você, quais têm sido os desafios e os principais aprendizados com OKRs? O que achou do OKR Canvas?

Compartilhe comigo sua opinião nos comentários abaixo!

Referências:

• Como definir OKRs em equipes ágeis?
• The Art of the OKR
• Blog do Felipe Castro

The post OKR: lições aprendidas para você começar a aplicá-lo de forma efetiva first appeared on Plataformatec Blog.

Trying to get a better understanding of this subject to talk to clients and stakeholders in general, I decided to collect real project data and analyze the relation between amount of story points of the cards and the time it took for each to be developed, counting since the time they started to be developed until the moment they are delivered to production, the famous Lead Time.

Disclaimers

Before we start, some disclaimers about the context of the project:

• Right in the first few weeks of the project we already had the first big problem with story points: subjectivity in estimate the cards. Why this card is worth 3? How do I know this card is twice or thrice the size of the X card? What do I consider in this estimative?
• Thus, it was suggested to the team to utilize some criteria to estimate, intending to reduce this subjectivity.
• My friend and colleague Henrique Oliveira suggested a Fibonacci-like matrix, taking into account T-Shirt Size estimates, to estimate the cards, that helped the team a lot. Check it below:

• Cards estimated as 13 points or higher were broken into smaller cards, as the only time we worked with a card with 13 points, it exceeded the timebox of the sprint, which was 10 business days. So we decide to do not let any card with 13 or more story points to be added to the sprint backlog.
• Cards were broken down into smaller cards in a way that they delivered something functional in production, so that the backend and frontend stayed together on the same card.

A look into the data

I began by collecting the data of the project as a whole and analyzing the lead time graph with all cards that had already been finished or that were in production.

After acquiring this data, I segmented them by the amount of story points and analyzed them individually, having the following graphs:

Cards with 1 story point

Cards with 2 story points

Cards with 3 story points

Cards with 5 story points

Cards with 8 story points

Cards with 13 story points

Data Analysis

After analyzing the data, I reached some conclusions:

• It’s possible to verify that cards worthing 3, 5 or 8 points have a development time (lead time) similar, not being possible to find a correlation between time and the number of story points (it is not scalar).
• If we take a category with plenty of samples, like the cards with 5 points, it’s evident they differ a lot, not being possible to make some affirmations such as “a card worth 5 points takes between X and Y days to finish” when X and Y a small interval of days (1 to 3 days). The same happens with cards with 3 and 8 points.

Observation: Data referring to cards worth 1, 2 or 13 story points were excluded from the individual analysis, because there was little data to be analyzed individually.

Other Analysis beyond story points x lead time

I reached more conclusions about the use of story points that I would like to share with you all:

• At the beginning of the project, when we started to use story points, the subjectivity was very high. By establishing some criteria for estimation, we were able to stabilize it, making the time of development vary a lot less, increasing predictability. If you want to develop this with your team, consider checking out this article from Mike Cohn.
• With the addition of more scoring criteria, like test complexity, dependency on people that are not in the team (internal or external to the company), risks related to the cards and more, we lowered the lead time variability and increased the card delivery rate by breaking the card into smaller ones.
• Many times people relate story points with only the development effort, and this makes the team consider only the time of the Execution of a task. Another point is that the testers/QAs and UXs/designers, have little to no involvement in the estimatives. Making the team think not only about development, but also about the duration of the process as a whole (considering the steps of code review, tests, deploy, etc) and inviting more people to share their concerns about the estimatives helps to improve the predictability of the team.
• With help from metrics like lead time breakdown and CFD, the process of stabilizing the scoring was much easier, as it made the team think about the whole process and showed in which step of the process the team was spending the most time with.

Conclusions

The major conclusions I had were:

• It wasn’t possible to relate time of development (lead time) with Story Points directly, as the goal of the study was.
• It is important to say Story Points should be used inside the team level. If you want to know more about this subject, I recommend the reading of this text from Troy Magennis.
• It was possible to increase the predictability of team delivery and stabilize the process of development using story points with other tools at the same time.
• There are indications that, when you add more criteria to the estimation to breakdown the cards into smaller cards, the smaller the risk/effort/complexity will be and, consequently, the predictability you also increase.
• An important point here is that story points and other estimative methods consider only the moments in which a demand is being worked on, and not consider/can’t predict the time that a demand will be staying in waiting to be worked on. Basically, the team considers how long it will take for a demand to be developed and tested, but don’t consider in this estimatives (as it is not possible to predict some points in the middle of the development) how long the demand will be staying in waiting to be tested, to be deployed or how long it will take to resolve a hypothetical impediment. In other words, the process efficiency will impact your lead time, and consequently, the relation between story points with the time of development. This way, it’s plausible that a user story of 1 story point takes the same amount of days that a 13 points user story to be delivered, because of the flow efficiency. If you would like to know more about process efficiency, check this article.

And if you were asked if you would use story points to estimate a deadline for a delivery of a project or card, what would you respond?

I would say that story points, like the matrix for complexity and uncertainty (T-shirt Size), are excellent tools to instigate your team in better breaking user stories down and help with improvements in the processes, however, to estimate deadlines, I prefer to utilize process metrics and other tools, like a Monte Carlo simulation to help predict when a set of cards will be delivered.

What about you? Do you have any experience with Story Points and time estimates? Have you passed through any similar scenario? What’s your opinion about what I shared in this text? Share with us commenting below or in this email contagil@plataformatec.com.br

The post Relation between Story Points and Development Time (Lead Time) first appeared on Plataformatec Blog.

A delay in a single week of a project creates two different cost types:

• The first is the cost of the team, since they will need to work another week.
• The second is the Cost of Delay, which is the income not collected due to the delay. For instance, suppose the new feature this team is working on would generate a daily income of $100, the Cost of Delay would be $700 ($100 * 7 days).

Since these costs can be high, using tools that provide more visibility on delivery dates is necessary. One tool that can help solve this problem is the Gantt Chart. With the Gantt Chart it’s possible to identify the most frail points of the project (the critical path) where more effort should be made to prevent delays, since a delay in any step of those points will impact the project as a whole.

Even though this tool is great for some types of projects (normally when there is a low uncertainty level), it’s not ideal for projects where the predictability is low due to the variance of the deliverables, as in projects that happen in the field of knowledge (writing a book or software development, for example).

Here at Plataformatec, we have always done our best to make our delivery predictions based on data and ensure they follow proven scientific methodology. The two methods we use the most are:

• Linear Progression
• Monte Carlo Simulation

LINEAR PROGRESSION

In linear progression, we do data analysis on the delivered work items during some time period (in software development we normally work with weeks) and through the study of this information we come up with the best values for the progression. For example: suppose we have the following historic values of work items by week, as shown below:

A value that could be used as a pessimistic projection would be to assume a delivery of 1 work item per week, seeing as we’ve had 5 cases where the throughput was 1 or 0. For the optimist projection we could use either the value of 3 or 4, or even 3.5, as we’ve had 4 instances where the throughput was 3 or higher. Lastly, for the likely projection, a good value would be 2, as it’s the median and the mode of this dataset.

The biggest problem with the linear progression is that we are inputting the values for each scenario, something that can be dangerous if the person operating the linear progression doesn’t have a good grasp of data analytics. Furthermore, linear progression ignores variance. Thus, in work systems with high variance of deliveries, the linear progression tool might not be the best approach.

MONTE CARLO

Another technique we can use is called Monte Carlo, which runs random picks from our historic throughput data to try and find the likelihood of delivery.

I’m having success using this method, however a question that keeps coming to my mind is: “How many interactions are necessary to make the method statistically valuable and the results reliable?”. Looking for an answer to this question, I’ve researched some statistics books, and also the internet, but wasn’t able to find any information that would help.

So I decided to run a study on this subject. Monte Carlo’s objective is to infer the probability of an event, “forcing” it to happen so many times, in order to use the law of large numbers in our favor. For example, if you flip a coin a thousand times, count how many times it falls as “heads”, and divide by a thousand, you will get an inference of the probability of getting “heads”.

To check the quality of the method, I first ran tests over probabilities for which I already knew the expected results. After that, I started increasing the complexity of the problem, in order to see if there was any correlation between the difficulty of a problem and the number of interactions necessary for the inferred value from Monte Carlo to be as close as possible to the calculated one.

The objective of the tests is to see how many interactions are necessary for the inferred value to be close enough to the calculated value that the gain of running more interactions would be too low to compensate the computer power necessary to run it (for a project probability standpoint an error in the first or second decimal values would be acceptable). I’d like to highlight that this blog post is not a scientific study of the subject, and the objective is to understand through inference a good value for project predictions.

THE TESTS

The tests were performed using the programing language R, which can be downloaded in this link. If you would rather use an IDE, I recommend using RStudio.

In each test, I’ve run the algorithm 100 times using the following interaction values: 100 (one hundred), 1000 (one thousand), 10000 (ten thousand), 100000 (one hundred thousand), 1000000 (one million), 10000000 (ten million), 100000000 (one hundred million) e 200000000 (two hundred million). The results of these 100 “rounds” are then consolidated.

THE COIN

The first test that I’ve run was the flipping of a single coin. In a coin flip, there is a 50% chance of it landing on any side, and this is the value we want the Monte Carlo to infer. For that I’ve used the following R code:

FLipCoin  <-  function(iterations)
{
  # Creates  a  vector  with  the  possible  values  for  a  coin  1  =  Heads  and  0  =  Tails
  coin  =  c(0,1)
  result  =  0

  # Flips  a  number  of  coins  equal  to  the  interaction  value  and  sum  up  all  the  times  where  the  coin  landed  in  "Heads"
  result  =  sum(sample(coin,  iterations,  replace=T))

  # Turns  the  value  in  a  percentage  of  the  coin  flips
  result  =  (result/iterations)  *  100

  return(result)
}

# Initiates  variables
result_vector  =  0
time_vector  =  0
control  =  0

# Controls  the  interaction  quantity
iterations  =  100

# Alocates  the  percentual  of  "Heads"  results  in  a  100  size  vector  and  the  execution  time  in  another
while(control  !=  100)
{
  start_time  =  Sys.time()
  result_vector  =  append(result_vector,  FLipCoin(iterations))  
  finish_time  =  Sys.time()

  time  =  finish_time  -  start_time
  time_vector  =  append(time_vector,  time)  

  control  =  control  +  1
}

# Shows  the  percentual  of  "Heads"
result_vector

# Shows  the  execution  times
time_vector

Code language: PHP (php)

Changing the values of the iterations and compiling the results, I’ve got the following table:

• Iterations	Min result	Max result	Expected result	Average result	Average result - Expected result	Result's median	Result's median - Expected	result | Result's Standard deviation
  100	33.00000	67.00000	50.00000	50.02000	0.02000	50.00000	0.00000	5.09368
  1000	43.60000	56.20000	50.00000	49.99440	0.00560	50.00000	0.00000	1.59105
  10000	48.35000	51.78000	50.00000	50.01263	0.01263	50.02000	0.02000	0.51001
  100000	49.58000	51.78000	50.00000	49.99110	0.00890	49.99350	0.00650	0.15805
  1000000	49.85170	50.15090	50.00000	49.99883	0.00117	49.99785	0.00215	0.04811
  10000000	49.95433	50.05807	50.00000	50.00013	0.00013	50.00010	0.00010	0.01564
  100000000	49.98435	50.01637	50.00000	50.00004	0.00004	49.99989	0.00011	0.00516
  200000000	49.98890	50.01195	50.00000	49.99981	0.00019	49.99987	0.00013	0.00345

In the instance of a simple problem, like the flipping of a single coin, we can see that a good iteration value could be:

• between 10M and 100M if you are ok with having an error in the third decimal,
• between 100k and 1M in the second decimal
• between 1k and 10k in the first decimal.

ONE DICE

With the coin test it was possible to infer the number of necessary interactions, based on the desired degree of reliability. However a question remains regarding if this behavior changes for more complex problems. Because of that, I’ve run a similar test with a single six-sided die that has a 16,67% chance of landing on any side.

RollDie  <-  function(iterations)
{
  # Creates  a  vector  with  the  possible  values  of  the  die
  die  =  1:6
  result  =  0

  # Alocates  the  die  roll  results
  result  =  sample(die,iterations,replace=T)

  # Sum  up  every  instace  where  the  die  landed  on  6
  result  =  sum(result  ==  6)

​
  # Turns  the  value  in  a  percentage  of  the  die  rolls
  result  =  (result/iterations)  *  100

  return(result)
}

# Initiates  variables
result_vector  =  0
time_vector  =  0
control  =  0

# Controls  the  interaction  quantity
iterations  =  100

# Alocates  the  percentual  of  "6"  results  in  a  100  size  vector  and  the  execution  time  in  another
while(control  !=  100)
{
  start_time  =  Sys.time()
  result_vector  =  append(result_vector,  RollDie(iterations))  
  finish_time  =  Sys.time()

​
  time  =  finish_time  -  start_time
  time_vector  =  append(time_vector,  time)  

  control  =  control  +  1
}

# Shows  the  percentual  of  "6"
result_vector

# Shows  the  execution  times
time_vector

Code language: PHP (php)

Changing the values of the iterations and compiling the results , I’ve got the following table:

We can see that in this case, the iteration values are the same as for the instance of the coin toss, where a good value could be something between 10M and 100M if you need to have an error in the third decimal, between 100k and 1M for an error in the second decimal and between 1k and 10k for an error in the first decimal.

This test result shows that, either the complexity of the problem has no influence on the number of interactions or that both problems (coin toss and single die roll) have similar complexity levels.

Analyzing the execution times for both problems, it’s possible to see that they took about the same time. This also corroborates with the hypothesis that they have similar complexity levels.

TWO DICE

To test the hypothesis that the complexity of the above problems were too close, and this was the reason for the interaction values to be close, I’ve decided to double the complexity of the die roll problem by rolling two dice instead of only one (it’s important to note that doubling the quantity of functions doesn’t always mean that the complexity order will double as well, as it depends on the Big-O Notation). The highest probability value for two dice is 7, with a 16,67% chance of occurrence.

RollsTwoDice <- function(iterations)
{
  # Creates a vector with the possible values of the die
  die = 1:6
  result = 0
	
  # Alocates the die roll results
  first_roll = sample(die,iterations,replace=T)
  second_roll = sample(die,iterations,replace=T)

  # Sum the vector values position-wise
  result = first_roll + second_roll

  # Sum up every instace where the dice landed on 7
  result = sum(result == 7)

  # Turns the value in a percentage of the dice rolls
  result = (result/iterations) * 100

  return(result)
}

# Initiates variables
result_vector = 0
time_vector = 0
control = 0

# Controls the interaction quantity
iterations = 100

# Alocates the percentual of "7" results in a 100 size vector and the execution time in another
while(control != 100)
{
  start_time = Sys.time()
  result_vector = append(result_vector, RollsTwoDice(iterations)) 
  finish_time = Sys.time()

  time = finish_time - start_time
  time_vector = append(time_vector, time) 

  control = control + 1
}

# Shows the percentual of "7"
result_vector

# Shows the execution times
time_vector
Code language: PHP (php)

Changing the values of the iterations and compiling the results, I’ve got the following table:

Again we find the same interaction quantity for each error level. In the case of this problem it’s possible to identify that it was more complex since the execution for 200M took on average 7.47 seconds longer when compared to the single die roll.

FIVE DICE

The inference that I’ve built until this point is that the complexity of the problem influences very little in the iteration quantities, but to be sure of that, I ran one last test with 5 dice. The highest probability when rolling five dice is the values of 17 and 18, with 10.03% probability each.

Since this problem is a lot more complex than the others I’ve decided to run only 10 times each instead of 100.

RollsFiveDice <- function(iterations)
{
  # Creates a vector with the possible values of the die
  dado = 1:6
  result = 0
	
  # Alocates the die roll results
  first_roll = sample(dado,iterations,replace=T)
  second_roll = sample(dado,iterations,replace=T)
  terceira_rolagem = sample(dado,iterations,replace=T)
  quarta_rolagem = sample(dado,iterations,replace=T)
  quinta_rolagem = sample(dado,iterations,replace=T)

  # Sum the vector values position-wise
  result = first_roll + second_roll + terceira_rolagem + quarta_rolagem + quinta_rolagem

  # Sum up every instace where the dice landed on 18
  result = sum(result == 18)

  # Turns the value in a percentage of the dice rolls
  result = (result/iterations) * 100
	
  return(result)
}

# Initiates variables
result_vector = 0
time_vector = 0
control = 0

# Controls the interaction quantity
iterations = 10

# Alocates the percentual of "18" results in a 100 size vector and the execution time in another
while(control != 100)
{
  start_time = Sys.time()
  result_vector = append(result_vector, RollsFiveDice(iterations)) 
  finish_time = Sys.time()

  time = finish_time - start_time
  time_vector = append(time_vector, time) 

  control = control + 1
}

# Shows the percentual of "18"
result_vector

# Shows the execution times
time_vector
Code language: PHP (php)

Changing the values of the iterations and compiling the results, I’ve got the following table:

Once again we can see that the values for the iterations are the same as above. It’s possible to notice that this problem is clearly more complex, as it took 35 seconds longer when compared to the roll of a single die.

CONCLUSION

After running those tests, it’s possible to conclude that for the use on project management, a good iteration value would be somewhere between 1k and 100M, depending on the error level as shown below:

• Error in the first decimal: 1k to 10k
• Error in the second decimal: 100k to 1M
• Error in the third decimal: 10M to 100M

The value that I’ve been using is 3M, for stakeholder reports. However, when I’m running the method for my own analysis, I use a lower value, somewhere between 250k and 750k.

How about you? Have you tried to run Monte Carlo Simulations? How many iterations are you using? Tell us in the comment section below or on Twitter @plataformatec.

The post Monte Carlo in Practice: Finding the ideal iteration value first appeared on Plataformatec Blog.

This sharing often leads to confusion. Do they provide distinct behaviors? Do they overlap? For instance, is there any purpose to Elixir’s fault tolerance if Kubernetes also provides self-healing?

In this article, I will go over many of these topics and show how they are mostly complementary and discuss the rare case where they do overlap.

Self-healing

Kubernetes automatically restarts or replaces containers that fail. It can also kill containers that don’t respond to your user-defined health check. Similarly, in Erlang and Elixir, you structure your code with the help of supervisors, which automatically restart parts of your application in case of failures.

Kubernetes provides fault-tolerance within the cluster, Erlang/Elixir provide it within your application. To understand this better, let’s take an application that has to talk to a database (or any other external system). Most languages handle this by keeping a pool of database connections.

If your database goes offline, because of a bad configuration or a hardware failure, both the database and the Erlang/Elixir systems will respond negatively to health checks, which would cause Kubernetes to act and potentially relocate them. This is a node-wide failure and Kubernetes got your back.

However, what happens when part of your connections to the database are sporadically failing? For example, imagine your system is under load and you suddenly started running into connection limits, such as MySQL’s prepared statement limit. This failure likely won’t cause any health check to fail but your code will fail whenever one of its many connections reach said limit. Can you reason about this error today in your applications? Can you confidently say that the faulty connection will be dropped? Will another connection be started in place of the faulty one? Can you comfortably say this error won’t cascade in the application bringing the remaining of the connection pool down?

Erlang/Elixir’s abstractions for fault tolerance allow you to reason about those questions at the language level. It provides a mechanism for you to reason about connections, resources, in-memory state, background workers, etc. You can explicitly say how they are started, how they are shut down, and what should happen when things go wrong. These features can also be extremely helpful in face of partial failures. For example, imagine you have a news website and the live stock ticker is down. Should the website continue running, potentially serving stale data, or should everything crash down? The mental model provided by Erlang/Elixir allows us to reason about these scenarios. And of course, you can always let failures bubble up after a few retries, or even immediately, so it becomes a node-wide failure to be handled by K8s.

In a nutshell, Kubernetes and containers provide isolation and an ability to restart individual nodes when they fail, but it is not a replacement for isolation and fault handling within your own software, regardless of your language of choice. Using K8s and Erlang/Elixir allow you to apply similar self-healing and fault-tolerance principles in the large (cluster) and in the small (language/instance).

Service discovery and Distributed Erlang

The Erlang VM also provides Distributed Erlang, which allows you to exchange messages between different instances running on the same or different machines. In Elixir, this is as easy as:

for node <- Node.list() do
  # Send :hello_world message to named process "MyProcess" in each node
  send {node, MyProcess}, :hello_world
end
Code language: PHP (php)

When running in distributed mode (which is not a requirement in any way and you need to explicitly enable it), the Erlang VM will automatically serialize and deserialize the data as well as make sure the connection between nodes is alive, but it does not provide any node discovery. It is the programmer responsibility to say exactly where each node is located and connect the nodes together.

Luckily, Kubernetes provides service discovery out of the box. This means that, K8s allows us to fully automate the node discovery, which would otherwise be manual and error prone. Libraries like libcluster do exactly that (and rolling your own wouldn’t be complicated either). This is another great example of where Kubernetes and the Erlang VM complement each other!

However, you may still be wondering, is there a benefit to running Distributed Erlang when Kubernetes’ Service Discovery makes it relatively easy to have systems communicating with each other? Especially when considering RPC protocols such as Thrift, gRPC, and others?

When we are talking about different languages and different systems communicating with each other, picking one of the existing RPC mechanisms is likely the best choice, and they will also work fine with Erlang/Elixir. The scenario where the Erlang VM really shines, in my opinion, is for building homogeneous systems, i.e. when you have multiple deployments of the same container and they exchange information. For example, imagine you are building a real-time application when you want to track which users are in the same chat room, or in the same city block, or in the same mountain track. As users connect and disconnect and as nodes are brought up and down, you could somehow update the database or communicate via a complex RPC mechanism, while carefully watching the cluster for topology changes.

With the Erlang VM, you can just broadcast or exchange this information directly, without having to worry about serialization protocols, connection management, etc, as everything is provided by the VM. All without external dependencies. This is one of the many features that makes Phoenix a breeze to build distributed web-realtime systems.

Automated rollouts vs Hot code swapping

When it comes to deployment, Kubernetes automatically rolls out changes to your application or its configuration, avoiding changing all instances at the same time. At the same time, the Erlang VM supports hot code swapping, which allows you to change the code that is running in production within a single instance without shutting said instance down.

Those two deployments techniques are obviously conflicting. In fact, hot code swapping does not go well in general with the whole idea of immutable containers. Does it mean that Kubernetes and the Erlang VM are a poor fit? Not really, because you don’t have to use hot code swapping. In fact, most people do not. Most Elixir applications are deployed using blue-green, canary, or similar techniques.

The truth about hot code swapping is that it is actually complicated to pull off in practice. Let’s use the database as an example once again. When you are deploying a new version of your software, whenever you update your database, you should never perform destructive changes. For example, if you want to rename a column, you have to add a new column, migrate the data over, and then remove the column. If you just rename the column, then you will have failures whenever doing rollouts, because you will have two versions of the software running at the same (one using the old column and the other using the new one). In hot code swapping, we have precisely the same issue, except it applies to all states inside your application. Companies that use hot code swapping often report they spend as much time developing the software as testing the upgrades themselves.

Of course, it doesn’t mean hot code swapping is useless. The Erlang VM development is mostly driven by business needs and there was a legitimate need for hot code swapping. In particular, when building telephone switches, there is never an appropriate moment to shut down an instance for updates, because at any given time a system is full of long running connections, perhaps days or even weeks. So being able to upgrade a live system is extremely helpful. If you have a similar need, then hot code swapping may be an option. Another option is to have smarter clients and migrate client connections between nodes when deploying.

Hot code swapping can also be used under other circumstances, such as during development to provide live code loading, without a need to restart your server, or to replace smaller components in production that don’t require replacing the whole instance.

Configuration management and Configuration providers

Another feature provided by both Elixir and Kubernetes is configuration management. However, as seen before, they work at very distinct levels. While Elixir provides a unified API for configuring applications, it is relatively low-level. In a production system, you often want both configuration and secrets to be managed by higher level tools, such as the ones provided by Kubernetes. Luckily, you can incorporate said configuration tools into your deployment workflow with the help of Configuration Providers. This functionality is part of Elixir releases, which were officially made part of the Elixir language in version 1.9.

Stay alert: pod resources

When provisioning Erlang and Elixir with Kubernetes, it is important to stay alert to one particular configuration: pod resources.

When using other technologies, it is common practice to break a large node into a bunch of small pods/containers. For example, if you have a node with 8 cores, you could allocate half of each CPU to a pod and split the memory equally between them, on a total of 16 pods.

This approach makes sense in many technologies that cannot exploit CPU and I/O concurrency simultaneously. However, the Erlang VM excels at managing system resources and your system will most likely be more efficient if you assign large pods to your Erlang and Elixir applications instead of breaking it apart into a bunch of small ones.

If the Erlang VM is sharing a machine with other applications you may want to consider reducing busy waiting. By doing so, the VM will optimize for lower CPU usage, making it a better neighbor, but with slightly higher latencies.

Summing up

Kubernetes and the Erlang VM work at distinct levels. Kubernetes orchestrates within a cluster, the Erlang VM orchestrates at the language level within an instance. Fred Hebert summed up this distinction well in a tweet:

Still seeing bad comparisons between kubernetes and #Erlang/OTP. K8s is to OTP what region failover is to k8s. They operate on different layers of abstraction and impact distinct components.

OTP allows handling partial failures WITHIN an instance, something k8s can’t help with.— Fred Hebert (@mononcqc) April 29, 2019

If you are using Erlang/Elixir and you wonder how Kubernetes applies compared to other languages, you can use Kubernetes for the Erlang VM as you would with any other technology. Given that Erlang/Elixir software can typically scale both horizontally and vertically, it gives you many options on how you want to allocate your resources within K8s.

On other areas, Kubernetes and the Erlang VM can nicely complement each other, such as using K8s Service Discovery to connect Erlang VM instances. Of course, Distributed Erlang is not a requirement and Erlang/Elixir are great languages even for stateless apps, thanks to its scalability and reliability.

If you are one of the few who really need hot code swapping in production, then the Erlang VM may be one of the best platforms to do so, but keep in mind you will be straying away from the common path in both technologies.

Finally, if you appreciate Kubernetes and its concepts, you may enjoy working with Erlang and Elixir, as they will give you an opportunity to apply similar idioms on the small and on the large.

Thanks to Fernando Tapia Rico, Fred Hebert, George Guimarães, Tristan Sloughter, and Wojtek Mach for reviewing this article.

The post Kubernetes and the Erlang VM: orchestration on the large and the small first appeared on Plataformatec Blog.