Improve your test readability using the xUnit structure

TL;DR: test quality is not just about verifying correctly whether your code works, it’s also about making your test easy to read and understand. You can do that by structuring your test using the four-phases xUnit standard.

The 4 phases of a test: setup, exercise, verify and teardown

People don’t write tests to be read, they write them to be executed

One of the main reasons to write tests is to have an automated way to check if your code is doing what you expect it to do. That means, trying to verify its correctness. Your test suite acts as a safety net that guarantees your software will continue to work as expected while you refactor, build new features or fix bugs. That’s amazing! But, throughout the years, software developers discovered that tests can be even more than a safety net.

Write tests as examples of how to use your code

A test is an example of how to use your code, not just a way to verify its correctness. Seeing tests as examples of how to use your code changes a little bit the priorities you have when writing them. If the test should serve as an example, then it should be easy to read and understand. Therefore, you should also focus on test readability, not just test “executability” (I know, weird word). One way to improve readability of a piece of text (or code) is to write it in a way the readers are used to, a structure that they expect, some standard way… Let’s think about that.

Back at your school days, you learned that when writing an essay, you’re supposed to structure it in: introduction, body and conclusion. Why? Because that structure helps you to better express your ideas. That means, it helps the reader to understand your message. Is there any equivalent of that for automated tests writing? In fact, there is. It’s called the xUnit structure.

Structure your tests using the xUnit standard

First, let’s see a test that can have its readability improved:

describe Stack do
  describe "#push" do
    it "puts an element at the top of the stack" do
      stack = Stack.new
      stack.push(1)
      stack.push(2)
      expect(stack.top).to eq(2)
    end
  end
end

One can understand the test above, but still, it’s not easy to quickly scan the test and see that it has logical parts. Those parts would be the the xUnit phases.

The standard xUnit test structure is composed of 4 phases: setup, exercise, verify and teardown.

  • Setup: this where you put the object under test in the necessary state for the behavior you want to check;
  • Exercise: when you send a message to your object;
  • Verify: here, you should check if the object under test behaved the way you expected;
  • Teardown: basically where you clean up stuff in order to get your system back to the initial state.

Now, let’s re-organize the test above making explicit that there are different logical parts:

describe Stack do
  describe "#push" do
    it "puts an element at the top of the stack" do
      # setup
      stack = Stack.new

      # exercise
      stack.push(1)
      stack.push(2)

      # verify
      expect(stack.top).to eq(2)
    end
  end
end

It’s easier to scan, isn’t it?

About the comments, no, we don’t need them. I added them in order to make the example clear. Let’s remove them and keep this structure:

describe Stack do
  describe "#push" do
    it "puts an element at the top of the stack" do
      stack = Stack.new

      stack.push(1)
      stack.push(2)

      expect(stack.top).to eq(2)
    end
  end
end

One can say that we just added two line breaks, that’s true. But that’s just the how, not the what. The what is: improving test readability. The how is: structuring the code based on the xUnit four-phase standard, by adding two line breaks. Got it?

Using a standard structure to ease the communication of an idea is not something new. As an example, Rails does that when it generates a standard directory structure. When entering on a new Rails project and scanning it, you know your way and where stuff are because you already expect a defined structure and you are used to it. It’s not something completely new, you’re used to that structure. I could also say that even Ruby uses that concept when it talks about the “principle of least surprise”, but maybe I would be going too far. So, let’s get that wrapped up.

Why care about test readability?

So, why should I care about all of that stuff? I mean, isn’t just having my test suite on green enough? No.

Test readability will be really important in a lot of situations. Like when a test gets red, someone needs to fix it. In order to do that, one needs to understand what the test is about. If the test is well structured and easy to read, they can fix it faster.

Also, if you think about your tests as examples of how to use your code, someone that is trying to use a class that you wrote, can see how it’s done in the tests. The test readability will be equally important here too.

So, what about you, how do you improve your test’s quality? How do you improve your test’s readability?

  • http://www.luizpicolo.com.br luizpicolo

    This is a great practice. Indeed, this simple separation of the code already provides a great read to understand

  • Felipe Rinaldi

    Automated tests is a great practice, specially to check code regressions in every new software build. If I could add my 2 cents, it is also great to do some manual alpha testing. Mess up with the product, specially new features. Mess with inputs, try unusual use cases, mess with the order you do things, click buttons you’re not expected to click, overload your system, see if the time it takes is reasonable, be very criterious of subjective aspects of your software and take notes constantly of how your software could improve, even if it works satisfactorily. I believe this practice can greatly improve the quality of your deliveries, now and in the future.

  • karlentwistle

    How about this? http://pastie.org/8988937

  • hugobarauna

    I understand the motivation and style, but I would prefer not going this direction, let me tell you why.

    A test is readable and clear when the reader can understand the relation of cause and effect from the setup to the verify phase. That said, the more test phases are contained inside the test, the better.

    Since we’re using RSpec, the ideal thing the do is putting every test phase inside the `it` block:


    it "puts an element at the top of the stack" do
    # setup
    # exercise
    # verify
    end

    There are even some test smells that are related to not having everything inside your test, like the mystery guest one.

  • https://about.me/jeff_dickey Jeff Dickey

    But if you discover a manual test that yields interesting results, such as a failure of the code under test, you should automate that and add it to your test suite. Tests are useless at best unless they’re repeatable, just as code is unknown at best until it’s been proven to work as expected. Technical debt is a trailing indicator of project failure; deal with it before you hit bankruptcy (project failure). Full test coverage also allows you to refactor at will and prove that your refactoring didn’t break anything. Without full coverage, you’ve only proven that any bugs are in the code that hasn’t been proven to work.

  • http://maurogeorge.github.com/ Mauro George

    Awesome post! I really understand your points, Joe Ferris talked about this, using the let[1] as the problem.

    Today I follow more like Myron[2], like the karlentwistle showed.

    In your example you showed a simple test, and whem I have lot of
    tests of the same method. I will duplicate the setup and exercise? Or
    create a helper mehtod?

    About the Mystery Guest, Joe Ferris uses simple helper methods, but is no real difference between a let and a helper that not receive a single param. This 2 guys, probably are a mistery guest, since all are defined outiside the test example, the difference is one uses the RSpec DSL.

    I guess it is all about tradeoffs we can repeat, this way we are not DRY and are without any Mistery Guest.

    Or we can have a Mistery Guest and be DRY.

    Today I really prefer be DRY.

    [1]: http://robots.thoughtbot.com/lets-not
    [2]: http://stackoverflow.com/questions/5359558/when-to-use-rspec-let/5359979#5359979

  • hugobarauna

    I will duplicate the setup and exercise? Or
    create a helper method?

    I prefer to remove duplications just from the setup phase. If all a test contain is the verify phase, it can be difficult to understand the cause and effect relationship.

    The worst thing that can happen to a test is not the it isn’t following DRY, it is not making the cause and effect relationship explicit and easy to understand.

    But as you yourself said, it’s all about tradeoffs. But, when thinking in tradeoffs in the tests world, I don’t usually favor DRY, I favor readability.