Three tips to improve the performance of your test suite

We, Rails developers, have always worried about improving the performance of our test suites. Today I would like to share three quick tips we employ in our projects that can drastically speed up your test suite.

1. Reduce Devise.stretches

Add the following to your spec/test helper:

Devise.stretches = 1

Explanation: Devise uses bcrypt-ruby by default to encrypt your password. Bcrypt is one of the best choices for such job because, different from other hash libraries like MD5, SHA1, SHA2, it was designed to be slow. So if someone steals your database it will take a long time for them to crack each password in it.

That said, it is expected that Devise will also be slow during tests as many tests are generating and comparing passwords. For this reason, a very easy way to improve your test suite performance is to reduce the value in Devise.stretches, which represents the cost taken while generating a password with bcrypt. This will make your passwords less secure, but that is ok as long as it applies only to the test environment.

Latest Devise versions already set stretches to one on test environments in your initializer, but if you have an older application, this will yield a nice improvement!

2. Increase your log level

Add the following to your spec/test helper:

Rails.logger.level = 4

Explanation: Rails by default logs everything that is happening in your test environment to “log/test.log”. By increasing the logger level, you will be able to reduce the IO during your tests. The only downside of this approach is that, if a test is failing, you won’t have anything logged. In such cases, just comment the configuration option above and run your tests again.

3. Use shared connection with transactional fixtures

If you are using Capybara for javascript tests and Active Record, add the lines below to your spec/test helper and be sure you are running with transactional fixtures equals to true:

class ActiveRecord::Base
  mattr_accessor :shared_connection
  @@shared_connection = nil

  def self.connection
    @@shared_connection || retrieve_connection

# Forces all threads to share the same connection. This works on
# Capybara because it starts the web server in a thread.
ActiveRecord::Base.shared_connection = ActiveRecord::Base.connection

Explanation: A long time ago, when Rails was still in 1.x branch, a new configuration option called use_transactional_fixtures was added to Rails. This feature is very simple: before each test Active Record will issue a begin transaction statement and issue a rollback after the test is executed. This is awesome because Active Record will ensure that no data will be left in our database by simply using transactions, which is really, really fast.

However, this approach may not work in all cases. Active Record connection pool works by creating a new connection to the database for each thread. And, by default, database connections do not share transactions state. This means that, if you create data inside a transaction in a thread (which has its own database connection), another thread cannot see the data created at all! This is usually not an issue, unless if you are using Capybara with Javascript tests.

When using Capybara with javascript tests, Capybara starts your Rails application inside a thread so the underlying browser (Selenium, Webkit, Celerity, etc) can access it. Since the test suite and the server are running in different threads, if our test suite is running inside a transaction, all the data created inside the test suite will no longer be available in the server. Alternatively, since the server is outside the transaction, data created by the server won’t be cleaned up. For this reason, many people turn off use_transactional_fixtures and use Database Cleaner to clean up their database after each test. However, this affects your test suite performance badly.

The patch above, however, provides a very simple solution to both problems. It forces Active Record to share the same connection between all threads. This is not a problem in your test suite because when the test thread is running, there is no request in the server thread. When the server thread is running, the test thread is waiting for a response from the server. So it is unlikely that both will use the connection at the same time. Therefore, with the patch above, you no longer need to use Database Cleaner (unless you are using another database like Mongo) and, more importantly, you must turn use_transactional_fixtures back to true, which will create a transaction wrapping both your test and server data, providing a great boost in your test suite performance.

Finally, if any part of your code is using threads to access the database and you need to test it, you can just set ActiveRecord::Base.shared_connection = nil during that specific test and everything should work great!


That’s it! I hope you have enjoyed those tips and, if they helped you boost your test suite performance, please let us know in the comments the time your test suite took to run before and after those changes! Also, please share any tips you may have as well!

15 responses to “Three tips to improve the performance of your test suite”

  1. Wojciech Wnętrzak says:

    Unfortunately #3 doesn’t work for me. I’m using rails 3.1.3, capybara 1.1.2, capybara-webkit 0.7.2 and rspec 2.7.0.
    On what kind of environment did you test it?

  2. josevalim says:

    I am certain that I have used it in exactly the following scenarios. Maybe it is something specific to your application that it is affecting it (for instance, threads or more than one connection to the database). Some sanity questions:

    1) Are you sure that the patch is being added before any of your tests run? 2) Are you sure you turned transactional fixtures back to true?

  3. Anonymous says:

    How can i use shared connection with spork?

  4. Very useful post, thanks.

    We recently shaved a minute or so off our test suite by getting rid of as many rake tasks as possible. We used to do:

    rake db:setup
    rake spec

    in about 6 different projects, and have changed this to just:

    rspec spec/

    We then use an in memory sqlite and just “require /path/to/schema.rb” in spec_helper.

    For some reason the overhead of running a rake task was quite big – any ideas why?

  5. Wojciech Wnętrzak says:

    Thanks for the tip.
    Indeed, in this application we’re launching another server (in separate thread), to do some heavy testing 😉

  6. josevalim says:

    When you do “rake spec”, you are starting a rake task in the development environment. Since RSpec needs to run in the test environment, it needs to start a new process. The starting up of processes (and therefore booting Rails several times) is the slow part. About the sqlite3 part, running it in memory and requiring db/schema.rb is a good strategy, although in the long run, when you start to rely on DB features that are not supported by sqlite3, you have no other option than falling back to mysql or postgresql.

  7. josevalim says:

    Unsure about Spork support. But well, just try it. If I had to choose, I would apply the shared connection patch after Spork forks. For example, putting it inside spec/support/shared_connection.rb may just work.

  8. Anonymous says:

    Don’t work for me. When i put that patch at the bottom of prefork block i always get PGError: not connected.

  9. Pavel Penkov says:

    Does Devise.stretches really affects test suite performance? I guess it’s impossible to notice unless you’re encrypting millions of passwords.

  10. Andrew Djoga says:

    Also to speed up your Capybara JS specs you can turn off jquery animation (if you use jquery):

    $ = true;


    In this case all animations will immediately set elements to their final state when called, rather than displaying an effect.

  11. Andrew Djoga says:

    Also to speed up your Capybara JS specs you can turn off jquery animation (if you use jquery):

    $ = true;


    In this case all animations will immediately set elements to their final state when called, rather than displaying an effect.

  12. nicholasmott says:

    Try putting it in the each_run block. Worked for me, at least to get rid of the PGError. Still have to check if it will work properly with Capybara in each_run.

  13. Aaron Jensen says:

    Saved us 30 seconds on our test run. Some additional tips:

  14. Aaron Jensen says:

    Is it possible to use shared connections with Cucumber? Cucumber seems to rely on DatabaseCleaner.

  15. Hello! I just would like to give a huge thumbs up for the great info you have here on this post. I will be coming back to your blog.