Posts by Hugo Baraúna

This post is part of a collection of posts we’re publishing on the subjects of low internal software quality, refactoring and rewrite.

Not only physical matter deteriorates, software does too

It’s known that physical matter deteriorates. People accept that and have always dealt with it. What people don’t accept so easily is that software “deteriorates” too. Unlike physical matter, it doesn’t happen due to some physical or chemical phenomenon. It usually happens because of some business change or people change. Let me give you an example.

Imagine you’re leading the tech or product team of a startup; you’re the CTO. You already launched your product’s first version, and it was a success. Your business model was validated, and now you’re in a growth stage. That’s awesome! But it has its costs, and it brings a new set of challenges.

The first version of your product is working, but the codebase is not in the shape you’ll need from now on. Maybe your team’s velocity is not as good as it used be. Your team keeps complaining about the code quality. The CEO and the product director want new features, and your current projections will not meet the business needs.

It’s not uncommon that one of the main sources of all these problems is the poor quality of your product’s codebase. You may need a refactor1 or a rewrite.

When the codebase is not in good shape, everyone can get frustrated

If the internal quality of your product is not good, everyone becomes frustrated.

Your whole team, including developers, will get frustrated because they would like to ship features faster, but the current code quality and architecture are not helping.

The IT, product, and software departments suffer because they’re not able to meet the expectations of the other departments.

The customer also suffers because of frequent bugs, how long it takes for them to be resolved, and how long it takes new features to be launched.

You get the picture.

Identifying the symptoms

It’s the leader’s job (let’s say the CTO) to identify when a refactor or a rewrite is needed. In order to do that, he or she can look around for some symptoms, like the ones below:

  • Everything is hard: Almost every feature or bug fix your team needs to do is hard. It was not always like that. You remember the good old days when your team was fast and everything ran smoothly.
  • Slow velocity: Your team’s velocity decreased or is decreasing. When you were building the first version of your product, it was fast to develop a new feature, and your team used to build lots of them every iteration. Now it’s different.
  • Slow test suite: Your test suite takes 10x, 20x, 30x more time to run than before.
  • Bugs that don’t go away: Your team fixes a bug, then in a week or so it appears again. Every now and then your team is fixing a regression bug.
  • Your team is demotivated: Your team keeps complaining that working in the project is not as productive as it was in the past. A single person can’t build one feature alone; there are too many moving parts.
  • Knowledge silos: There are some parts of the software that only a single developer knows well enough to maintain. It’s difficult for the rest of the team to work with that specific code.
  • New developer ramp-up time is taking too long: When new developers join the team, it takes too much time for them to be fully productive.

The reason you got into one of these situations is probably not a technical one. Maybe you needed to deliver too much, too fast while you were building the first version of your product. Maybe your team didn’t have the maturity and experience in the past they have now. Analyzing the root cause is important too, but you need to do something else. You need to solve your problem.

If you’re experiencing the symptoms above, you probably have a low internal software quality problem. Recognizing the symptoms is already a big step. The next step is to think of solutions. Some solutions you may take include refactoring or a rewrite process.

Refactor or rewrite?

There’s no definitive guide about when you should do a big refactor or a rewrite, because it depends a lot on your context. That said, there are some rules of thumb that you should consider when evaluating which solution to go with:

When to rewrite

  • The technology you use is outdated, and it’s not maintained anymore.
  • Your software is really slow, and changing the architecture is not enough or is not viable.
  • The supply of software developers that know the technology you use is low and decreasing.
  • There are new technologies that offer a significant advantage compared to what you’re using.

When to refactor

  • The technology you use is still maintained and relevant.
  • It’s viable to improve your application in an incremental fashion.
  • The problem you’re solving is just technical and not a business one.

Choosing one of these options is not an easy decision, and once you go with one of them, there will be an entire new set of concerns you’ll encounter. Stay tuned, in our next blog posts we’ll talk about what to consider when doing a big refactor or a rewrite.

Now I would like to know about your experiences. Have you ever been in a similar situation? How did you identify that your problem was low internal software quality? Please share with us!


  1. I prefer the term “code refurbishment”, but people aren’t generally used to it. So I’ll use refactoring in this blog post for the sake of clarity. 

TL;DR: test quality is not just about verifying correctly whether your code works, it’s also about making your test easy to read and understand. You can do that by structuring your test using the four-phases xUnit standard.

The 4 phases of a test: setup, exercise, verify and teardown

People don’t write tests to be read, they write them to be executed

One of the main reasons to write tests is to have an automated way to check if your code is doing what you expect it to do. That means, trying to verify its correctness. Your test suite acts as a safety net that guarantees your software will continue to work as expected while you refactor, build new features or fix bugs. That’s amazing! But, throughout the years, software developers discovered that tests can be even more than a safety net.

Write tests as examples of how to use your code

A test is an example of how to use your code, not just a way to verify its correctness. Seeing tests as examples of how to use your code changes a little bit the priorities you have when writing them. If the test should serve as an example, then it should be easy to read and understand. Therefore, you should also focus on test readability, not just test “executability” (I know, weird word). One way to improve readability of a piece of text (or code) is to write it in a way the readers are used to, a structure that they expect, some standard way… Let’s think about that.

Back at your school days, you learned that when writing an essay, you’re supposed to structure it in: introduction, body and conclusion. Why? Because that structure helps you to better express your ideas. That means, it helps the reader to understand your message. Is there any equivalent of that for automated tests writing? In fact, there is. It’s called the xUnit structure.

Structure your tests using the xUnit standard

First, let’s see a test that can have its readability improved:

describe Stack do
  describe "#push" do
    it "puts an element at the top of the stack" do
      stack = Stack.new
      stack.push(1)
      stack.push(2)
      expect(stack.top).to eq(2)
    end
  end
end

One can understand the test above, but still, it’s not easy to quickly scan the test and see that it has logical parts. Those parts would be the the xUnit phases.

The standard xUnit test structure is composed of 4 phases: setup, exercise, verify and teardown.

  • Setup: this where you put the object under test in the necessary state for the behavior you want to check;
  • Exercise: when you send a message to your object;
  • Verify: here, you should check if the object under test behaved the way you expected;
  • Teardown: basically where you clean up stuff in order to get your system back to the initial state.

Now, let’s re-organize the test above making explicit that there are different logical parts:

describe Stack do
  describe "#push" do
    it "puts an element at the top of the stack" do
      # setup
      stack = Stack.new
 
      # exercise
      stack.push(1)
      stack.push(2)
 
      # verify
      expect(stack.top).to eq(2)
    end
  end
end

It’s easier to scan, isn’t it?

About the comments, no, we don’t need them. I added them in order to make the example clear. Let’s remove them and keep this structure:

describe Stack do
  describe "#push" do
    it "puts an element at the top of the stack" do
      stack = Stack.new
 
      stack.push(1)
      stack.push(2)
 
      expect(stack.top).to eq(2)
    end
  end
end

One can say that we just added two line breaks, that’s true. But that’s just the how, not the what. The what is: improving test readability. The how is: structuring the code based on the xUnit four-phase standard, by adding two line breaks. Got it?

Using a standard structure to ease the communication of an idea is not something new. As an example, Rails does that when it generates a standard directory structure. When entering on a new Rails project and scanning it, you know your way and where stuff are because you already expect a defined structure and you are used to it. It’s not something completely new, you’re used to that structure. I could also say that even Ruby uses that concept when it talks about the “principle of least surprise”, but maybe I would be going too far. So, let’s get that wrapped up.

Why care about test readability?

So, why should I care about all of that stuff? I mean, isn’t just having my test suite on green enough? No.

Test readability will be really important in a lot of situations. Like when a test gets red, someone needs to fix it. In order to do that, one needs to understand what the test is about. If the test is well structured and easy to read, they can fix it faster.

Also, if you think about your tests as examples of how to use your code, someone that is trying to use a class that you wrote, can see how it’s done in the tests. The test readability will be equally important here too.

So, what about you, how do you improve your test’s quality? How do you improve your test’s readability?

Here at PlataformaTec we like to use Capybara for acceptance tests. Recently we have discovered the custom selectors feature in Capybara and we would like to share with you how that feature helped us to improve our tests.

Sometimes we need to implement features that involves showing some ordered items to the user, like a ranking feature. The HTML for a feature like that could be:

<ol id="overall-ranking">
  <% @top_users.each do |user| %>
    <li><%= user.name %></li>
  <% end %>
</ol>

The acceptance tests for this ranking could be written as follows:

scenario "The user can see an overall ranking" do
  Factory(:user, :name => "Hugo",  :score => 5000)
  Factory(:user, :name => "Ozaki", :score => 3000)
  Factory(:user, :name => "João",  :score => 4000)
 
  visit overall_ranking_path
 
  within("#overall-ranking") do
    find(:xpath, './/li[1]').text.should match("Hugo")
    find(:xpath, './/li[2]').text.should match("João")
    find(:xpath, './/li[3]').text.should match("Ozaki")
  end
end

Generally, I don’t like to see those XPath selectors inside my acceptance tests. And sometimes it can get really ugly! So, in order to improve our tests, we can create a custom selector with Capybara as follows:

# spec/spec_helper.rb
 
RSpec.configure do |config|
 
  Capybara.add_selector(:li) do
    xpath { |num| ".//li[#{num}]" }
  end
 
end

After that, we can refactor our test as shown below:

scenario "The user can see an overall ranking" do
  Factory(:user, :name => "Hugo",  :score => 5000)
  Factory(:user, :name => "Ozaki", :score => 3000)
  Factory(:user, :name => "João",  :score => 4000)
 
  visit overall_ranking_path
 
  within("#overall-ranking") do
    find(:li, 1).text.should match("Hugo")
    find(:li, 2).text.should match("João")
    find(:li, 3).text.should match("Ozaki")
  end
end

If you wanna know more about Capybara’s custom selectors, check its README.

And you? Any tips about using Capybara or improving your acceptance/integration tests?

Nesse sábado (29/05/2010) eu fiz uma palestra sobre Rails 3 no evento Ruby e Rails no Mundo Real 2010. O evento foi muito bom, tendo cerca de 200 pessoas participando. Queria agradecer a todos que estavam lá para ver minha palestra e agradecer também aos elogios, muito obrigado.

Seguem meus slides:

E o vídeo da minha palestra, que foi feito pelo @agaelebe.

I’ve been playing a lot with Rails 3 lately, it’s completely awesome! However, we don’t have any Rails 3 app in production at the moment, all projects that Plataforma has been doing for its clients so far are using Rails 2.3.5. Since I was already having fun with Rails 3, I thought I should also give ruby 1.9 a try. Ok, so how could I have Rails 2.3.5, Rails 3, ruby 1.8.7 and ruby 1.9 in my machine without going crazy? The answer is RVM (Ruby Version Manager)!

RVM is an awesome tool that gives you the power to have as many rubies in your machine as you want, and the best of all, all versions are isolated, no conflicts at all. In order to install it, follow the instructions inside RVM’s site. After that, I installed ruby 1.9.1 through RVM with rvm install 1.9, installed all the gems necesssary to play with Rails 3 (gem install rails --pre) and then, I could start to play.

I was playing with that configuration successfully until the moment I got a segmentation fault error. Other people got the same error too. Thankfully the problem didn’t last too long, this was one of these moments which I was very happy for having a Rails Core member in my team (@josevalim). He told me that Rails 3 doesn’t support ruby 1.9.1, actually, it’s supporting ruby 1.9.2! So, I needed to install a new ruby version, and install all Rails 3 gems again, damn it! But, I was fortunate enough for being using RVM.

In order to install Ruby 1.9.2-head and Rails 3 gems, all I needed to do was:

  1. Update my RVM: rvm update --head
  2. Install ruby 1.9.2 head: rvm install 1.9.2-head
  3. Create a Rails 3 RVM gemset: rvm --create use 1.9.2-head@rails3
  4. Copy my already installed Rails 3 gems to my new RVM gemset within my ruby 1.9.2: rvm gemset copy 1.9.1 1.9.2-head@rails3

That’s all! After that, I was back to the game! Man, RVM is awesome!!!

Be sure to have a look at a Wayne’s gist about how to install Rails 3 with RVM, and, check out RVM’s Gemsets feature, it’s very useful.

And you, do you have any tips about dealing with many versions of ruby and/or Rails, or about RVM?

So, you already did the right choice of using Delayed Job for your background processing, great! But, how are you going to be certain that your background processing will still be happening while you are sleeping? And if your Delayed Job process goes down, are you going to wake up in the dawn e restart it manually? I wouldn’t do that, I really appreciate my sleep. So, what’s the solution?

As rubyists and railers, we already know there are solutions, like God, that do this job for us. However, there are another solutions, like Bluepill. Bluepill is a process monitoring tool like God, but, unlike God, it doesn’t have memory leak, according to its authors.

Well, as I don’t want to wake up in the dawn to restart my Delayed Job process, and neither I want to restart my God process because of memory leaking, I decided to use Bluepill. But, how I use Bluepill to monitor my Delayed Job process?

In order to configure Bluepill to monitor Delayed Job, and use Capistrano to automate some tasks, we have basically 4 steps:

  1. Install and configure Delayed Job
  2. Install and configure Bluepill
  3. Write a Capistrano Recipe for Bluepill
  4. Set some stuff in /etc/sudoers

Let’s take a look at each step.

Installing and configuring Delayed Job

Installing and configuring Delayed Job is super simple, just read the project’s README, which is very clear. I also recommend watching the RailsCast episode about Delayed Job, there is some good information there, like using the Delayed Job CollectiveIdea’s fork, instead of the original repo’s version.

Installing and configuring Bluepill

There’s no secret in installing Bluepill too, you just need to read the project’s README and follow its steps.

In the confiuration part, you can see all the options in the README too. In my case, the configuration file is at RAILS_ROOT/config/production.pill.

Bluepill.application("my_app") do |app|
  app.process("delayed_job") do |process|
    process.working_dir = "/home/deploy/my_app/current"
 
    process.start_grace_time    = 10.seconds
    process.stop_grace_time     = 10.seconds
    process.restart_grace_time  = 10.seconds
 
    process.start_command = "ruby script/delayed_job -e production start"
    process.stop_command  = "ruby script/delayed_job -e production stop"
 
    process.pid_file = "/home/deploy/my_app/shared/pids/delayed_job.pid"
    process.uid = "deploy"
    process.gid = "deploy"
  end
end

However, I had a little problem between the interaction of Bluepill and Delayed Job. Delayed Job is not interpreting very well the -e production flag. You can see more details about that in an issue I opened.

The first solution I thought was to use RAILS_ENV=production ruby script/delayed_job start, however, for some reason that I don’t know exactly, it didn’t work.

So, the solution I came up with was to modify the file in RAILS_ROOT/script/delayed_job to the following one:

1
2
3
4
5
6
7
8
9
#!/usr/bin/env ruby
 
# TODO improve the line of code below
# The line below is just a hack while we wait the delayed job guys answer the following issue
# http://github.com/collectiveidea/delayed_job/issues#issue/38
ENV['RAILS_ENV'] ||= "production"
require File.expand_path(File.join(File.dirname(__FILE__), '..', 'config', 'environment'))
require 'delayed/command'
Delayed::Command.new(ARGV).daemonize

As the -e production was not being interpreted properly by Delayed Job, I’m setting manually the RAILS_ENV to production (line 6), therefore I’m supposing that you are using Bluepill to monitor processes that are in a production environment.

Capistrano recipe for Bluepill

In order to automate the start and stop Bluepill’s tasks, I wrote the following capistrano’s recipe:

# deploy credentials
set :user, "deploy"
set :deploy_to, "/home/deploy/#{application}"
set :use_sudo, false
set :rails_env, "production"
 
# Bluepill related tasks
after "deploy:update", "bluepill:quit", "bluepill:start"
namespace :bluepill do
  desc "Stop processes that bluepill is monitoring and quit bluepill"
  task :quit, :roles => [:app] do
    sudo "bluepill stop"
    sudo "bluepill quit"
  end
 
  desc "Load bluepill configuration and start it"
  task :start, :roles => [:app] do
    sudo "bluepill load /home/deploy/my_app/current/config/production.pill"
  end
 
  desc "Prints bluepills monitored processes statuses"
  task :status, :roles => [:app] do
    sudo "bluepill status"
  end
end

Note that instead of using the run capistrano’s method, I’m using the sudo method. I’m doing this because the bluepill command must be run as root. And that, takes us to the next topic.

/etc/sudoers

Since I need to run the bluepill command as root, should I change from set :user, "deploy" to set :user, "root"? I think it’s not a good idea, we don’t like to give root access to anything, even to deployment. So, what should I do? It’s simple, you just need to edit your sudoers file.

In order to do this, you need to use the visudo command to open and edit the /etc/sudoers file. Once with the file opened, just add the following line to the end of the file:

deploy ALL=(ALL) NOPASSWD: /usr/local/bin/bluepill

Now, you’re done, the deploy user already can do sudo bluepill without giving any password. Problem solved without opening security holes.

After these 4 steps, you’re ready to sleep at night without worrying about your Delayed Job process. And, if you want to know the status of the monitored processes by Buepill, you just need to run the following capistrano task in your local machine:

cap bluepill:status

And you, what are your solutions to sleep in peace?

Update: if are having trouble with restarting Bluepill with Capistrano, take a look at this.