Posts by José Valim

We, Rails developers, have always worried about improving the performance of our test suites. Today I would like to share three quick tips we employ in our projects that can drastically speed up your test suite.

1. Reduce Devise.stretches

Add the following to your spec/test helper:

Devise.stretches = 1

Explanation: Devise uses bcrypt-ruby by default to encrypt your password. Bcrypt is one of the best choices for such job because, different from other hash libraries like MD5, SHA1, SHA2, it was designed to be slow. So if someone steals your database it will take a long time for them to crack each password in it.

That said, it is expected that Devise will also be slow during tests as many tests are generating and comparing passwords. For this reason, a very easy way to improve your test suite performance is to reduce the value in Devise.stretches, which represents the cost taken while generating a password with bcrypt. This will make your passwords less secure, but that is ok as long as it applies only to the test environment.

Latest Devise versions already set stretches to one on test environments in your initializer, but if you have an older application, this will yield a nice improvement!

2. Increase your log level

Add the following to your spec/test helper:

Rails.logger.level = 4

Explanation: Rails by default logs everything that is happening in your test environment to “log/test.log”. By increasing the logger level, you will be able to reduce the IO during your tests. The only downside of this approach is that, if a test is failing, you won’t have anything logged. In such cases, just comment the configuration option above and run your tests again.

3. Use shared connection with transactional fixtures

If you are using Capybara for javascript tests and Active Record, add the lines below to your spec/test helper and be sure you are running with transactional fixtures equals to true:

class ActiveRecord::Base
  mattr_accessor :shared_connection
  @@shared_connection = nil
 
  def self.connection
    @@shared_connection || retrieve_connection
  end
end
 
# Forces all threads to share the same connection. This works on
# Capybara because it starts the web server in a thread.
ActiveRecord::Base.shared_connection = ActiveRecord::Base.connection

Explanation: A long time ago, when Rails was still in 1.x branch, a new configuration option called use_transactional_fixtures was added to Rails. This feature is very simple: before each test Active Record will issue a begin transaction statement and issue a rollback after the test is executed. This is awesome because Active Record will ensure that no data will be left in our database by simply using transactions, which is really, really fast.

However, this approach may not work in all cases. Active Record connection pool works by creating a new connection to the database for each thread. And, by default, database connections do not share transactions state. This means that, if you create data inside a transaction in a thread (which has its own database connection), another thread cannot see the data created at all! This is usually not an issue, unless if you are using Capybara with Javascript tests.

When using Capybara with javascript tests, Capybara starts your Rails application inside a thread so the underlying browser (Selenium, Webkit, Celerity, etc) can access it. Since the test suite and the server are running in different threads, if our test suite is running inside a transaction, all the data created inside the test suite will no longer be available in the server. Alternatively, since the server is outside the transaction, data created by the server won’t be cleaned up. For this reason, many people turn off use_transactional_fixtures and use Database Cleaner to clean up their database after each test. However, this affects your test suite performance badly.

The patch above, however, provides a very simple solution to both problems. It forces Active Record to share the same connection between all threads. This is not a problem in your test suite because when the test thread is running, there is no request in the server thread. When the server thread is running, the test thread is waiting for a response from the server. So it is unlikely that both will use the connection at the same time. Therefore, with the patch above, you no longer need to use Database Cleaner (unless you are using another database like Mongo) and, more importantly, you must turn use_transactional_fixtures back to true, which will create a transaction wrapping both your test and server data, providing a great boost in your test suite performance.

Finally, if any part of your code is using threads to access the database and you need to test it, you can just set ActiveRecord::Base.shared_connection = nil during that specific test and everything should work great!

Conclusion

That’s it! I hope you have enjoyed those tips and, if they helped you boost your test suite performance, please let us know in the comments the time your test suite took to run before and after those changes! Also, please share any tips you may have as well!

Last week I spoke at Silicon Valley Ruby Group about PlataformaTec’s open source tools, mainly Devise, Simple Form and Responders.

When talking about Devise, I’ve mentioned that, before creating Devise, we were alternating between using Authlogic or Clearance in our projects. However, we soon realized that we needed a solution that was as customizable as Authlogic (allowing us to choose behaviors and several configuration options) and as complete as Clearance (whole MVC stack). It is fun to remember this happened more than 2 years ago.

After the presentation, someone came to talk to me about Sorcery and said it would be nice if Devise provided the same kind of tooling, allowing someone to build their own controllers and views around Devise instead of using Devise built-in controllers and views. His proposal surprised me, because this approach is totally possible with Devise and it was one of our design goals since day one.

That said, we realized that we were probably not “advertising” the bare-bone, stripped-down aspect of Devise well enough. That’s why I am writing this blog post. Devise already makes it easy for you to customize your own views, using the generator rails g devise:views which copies the views to your application. But what if you want to roll out your own views AND controllers?

To show you how we can achieve that, let’s write some code! The first step is to create a Rails application:

rails new devise-only-model

Next, we will add Devise to the Gemfile:

gem "devise", "~> 1.4.6"

And run the installation generator:

bundle install && rails g devise:install

The installation generator is going to give you some extra instructions, so don’t forget to do that as well. Next, let’s generate our basic User model, but we will pass an extra parameter called --skip-routes:

rails g devise User --skip-routes

By passing this extra parameter, Devise is going to generate everything as usual, with a small difference on config/routes.rb:

devise_for :users, :skip => :all

This parameter tells Devise to not generate any route at all. You can check that by executing bundle exec rake routes. However, you may be wondering: why can’t we simply remove the devise_for call? If we remove the route, Devise wouldn’t actually know that you have added Devise configuration to the User model, as all models are lazy loaded. So we need the route to tell Devise it needs to setup the appropriate helpers for the user, like authenticate_user!.

With Devise configured, we are ready to create the controllers and views on our own. In this blog post, we are going to create the SessionsController as an example allowing us to sign in and sign out. First, let’s add our routes:

root :to => "sessions#new"
post "/users/sign_in"    => "sessions#create"
delete "/users/sign_out" => "sessions#destroy"

Our SessionsController at app/controllers/sessions_controller looks like:

class SessionsController < ApplicationController
  # For security purposes, Devise just authenticates an user
  # from the params hash if we explicitly allow it to. That's
  # why we need to call the before filter below.
  before_filter :allow_params_authentication!, :only => :create
 
  def new
    @user = User.new(params[:user])
  end
 
  def create
    # Since the authentication happens in the rack layer,
    # we need to tell Devise to call the action "sessions#new"
    # in case something goes bad. Feel free to change it.
    user = authenticate_user!(:recall => "sessions#new")
    flash[:notice] = "You are now signed in!"
    sign_in user
    redirect_to root_path
  end
 
  def destroy
    sign_out
    flash[:notice] = "You are now signed out!"
    redirect_to root_path
  end
end

The controller implementation is quite straightforward. With the controller in hands, we just need to generate the view for the new action at app/views/sessions/new.html.erb:

<% if user_signed_in? %>
  You are signed in as <%= current_user.email %>. <%= link_to "Sign out", users_sign_out_path, :method => :delete %>.
<% else %>
  <%= form_for @user, :url => users_sign_in_path do |f| %>
  <div><%= f.label :email %><br />
  <%= f.email_field :email %></div>
  <div><%= f.label :password %><br />
  <%= f.password_field :password %></div>
  <div><%= f.check_box :remember_me %> <%= f.label :remember_me %></div>
  <div><%= f.submit "Sign in" %></div>
  <% end %>
<% end %>

The view shows a message if the user is signed in, otherwise it shows a sign in form. Now we are almost ready to check if it works. First, we need to run the migrations:

bundle exec rake db:migrate

Remove the index page:

rm public/index.html

And create a user in the database so we can sign in. We can do that in rails console:

User.create!(:email => "jose@example.com", :password => "123456")

Now, start the server and you are ready to sign in and sign out. You can also block user access in any controller by calling authenticate_user! in a before filter. Just remember that, if you add the filter to your application controller, remember to skip the filter on the sessions controller, otherwise you won’t be able to sign in in the first place.

You can now freely proceed to implement the other controllers and views in your application. Keep in mind that if you have devise :recoverable in your model, all the related methods like User.send_reset_password_instructions will already be available in your model, so you can use them straight away to implement your own reset password feature. Since Devise use all those methods internally, if you have any questions about implementing your own reset password feature, you can always take a look at Devise own controllers for some help.

I hope this post can help you to roll out your own controllers if this is the kind of feature you expect from Devise. Also, if you are worried about the overhead of using Devise even if you are not using its controllers, there is no need to worry at all. Devise does the smart thing and only loads the controllers you are actually using. Also, it lazily loads all behaviors, so if you are not using recoverable, no code related to recoverable will be loaded at all.

It is important to keep in mind that Devise was built by us to be flexible and capable of handling different requirements from different clients, so it is PlataformaTec’s priority to have it as flexible as possible! So, what would you like to see in “bare-bone, stripped-down Devise” in order to better use it in your applications?

When writing Crafting Rails Applications, I knew exactly which parts from Rails I wanted to talk about. However, I didn’t want the book to simply describe how everything works, I actually wanted everyone to build something useful from each part of Rails.

One of the hardest areas to come up with an useful tool as example was the template handlers. Template handlers are responsible for template compilation and the canonical examples are: ERb and Haml. Obviously, creating something like ERb or Haml from scratch would require a lot of code beyond the Rails integration so it wasn’t an option. On the other hand, tools that simply render rdoc or markdown templates would be too simple and there are already plenty of gems doing the same out there.

So I started playing with some options. The first one was something like Why’s Markaby but that would still require a good amount of code (albeit much less than ERb). Next, I have played with something called YERb (YAML + ERb) which was an interesting hack but too slow to be of any use. I was almost planning to remove the chapter about template handlers when it finally came to me the idea of markerb: markdown + erb (in the book, we did a wordplay and called merb). While the idea was simple and easy to implement, it had a great use case: multipart templates.

With Markerb, you can create one template for ActionMailer and it will be delivered both as text and HTML. So there is no need to maintain two templates. You write it in markdown, which is delivered as text, but also rendered to be delivered as HTML. Recently, I have crafted Markerb in its own gem so everyone can use it.

How to use?

The usage is quite simple. Assuming you have a notifier as below:

class Notifier < ActionMailer::Base
  def contact(recipient)
    @recipient = recipient
    mail(:to => @recipient, :from => "john.doe@example.com") do |format|
      format.text
      format.html
    end
  end
end

If you create a template at app/views/notifier/contact.markerb:

Multipart templates **rock**, right <%= @recipient %>?!

It will generate two parts, one in text and another in HTML when delivered. And that is it! Before we finish, here are a few things you might need to know:

  • The “contact.markerb” template should not have a format in its name. Adding a format would make it unavailable to be rendered in different formats;
  • The order of the parts matter. It is important for e-mail clients that you call format.text before you call format.html;
  • Notice you can normally use ERb inside the template.

If you are interested in how Markerb, template handlers and other part of Rails work, check out Crafting Rails Applications. We also hope to add a generator to Devise that will optionally copy all views as Markerb templates to your application.

And you? Do you have any use cases for Markerb? Do you have any other interesting template handlers you would like to share?

It is common in Rails 3.0 applications that you want to provide default views for a group of controllers. Let’s say you have a bunch of controllers inside the Admin namespace and you would like each action to fallback to a default template. So if you are rendering the index action for Admin::PostsController and “app/views/admin/posts/index.html.*” is not available, it should then render “app/views/admin/defaults/index.html”.

There are several ways to implement this feature at the controller level. It mainly relies on trying to render the original template and then rescue ActionView::MissingTemplate. If this error is rescued, you then render the default one. However, there is a considerable performance overhead in this approach as it needs to pass through the rendering and template lookup stack twice.

Luckily, since Rails 3.0, we have a new abstraction called resolvers that holds the logic to find a template. I explain comprehensively how resolvers work and their API in my book Crafting Rails Applications. So here I would just show the basics to get this functionality working.
First, we need to define a DefaultResolver, it could be done inside the lib directory:

class MyResolver < ::ActionView::FileSystemResolver
  def initialize
    super("app/views")
  end
 
  def find_templates(name, prefix, partial, details)
    super(name, "admin/defaults", partial, details)
  end
end

Our new resolver simply inherits from ActionView::FileSystemResolver and does two changes: Overrides the initialize method so the view path defaults to “app/views” inside our application and overrides find_templates. The find_templates method receives the template name, a prefix (i.e. the controller path), a boolean marking if the template is a partial or not and a hash of details. In the example above, we simply ignore the prefix given and hardcode it to “admin/defaults”.

Now, assuming that all controllers inside the Admin namespace inherit from an Admin::ApplicationController, we can add default views to all of them by adding the following line:

class Admin::ApplicationController < ActionController::Base
  append_view_path MyResolver.new
end

And we are done! The view_paths holds a list of paths and/or resolvers that the controller will look for templates until one is found. If none is found, an ActionView::MissingTemplate is raised. Since we used append_view_paths, our resolver was added after the “app/views” path, used by default in all controllers.

As you may have guessed, resolvers are a powerful abstraction that allows you to retrieve templates from anywhere, including the database, which is the example given in Crafting Rails Applications.

Finally, template inheritance was a feature recently added to Rails master (upcoming Rails 3.1), so you won’t need to create your custom resolver as above. There is a good wrap up about this feature in Rails Edge.

Erik DeBill has put two interesting benchmarks on his blog. The first one compares the performance of different Ruby implementations in Rails development mode while the second compares their performance in Rails boot time. If you haven’t read them yet, please do it now.

Benchmarking code is an important practice, but it can be misleading if you fail to understand the root causes that lead to the different results.

Performance in development mode

In the first blog post, it is guessed that the root case for having slow requests in development is because Rails eager loads all models and controllers for each request:

Now, what I’d really like is a way to avoid recompiling everything every time. If I could have Rails recompile just the model or controller I’m working on and skip all the others, that’d be grand. I’ve taken a couple stabs at it, but I haven’t succeeded yet.

This is wrong! Rails, in development, only loads the model and the controller you are using in that specific request. This is very easy to verify if you create a new application, scaffold two resources and add a puts self.name in their class definition. If you access one controller, it will only load the model explicitly referenced in that controller. Even the model associations try to be lazy in that aspect, always loading the minimum it can.

So you may ask, why Rails is getting so slow after adding more scaffolds?

It happens because Rails 3.0 includes all helpers by default in your ApplicationController. So at the beginning of each request, Rails needs to load all helpers. Loading a helper in development mode is slow because ActiveSupport::Dependencies needs to track which dependencies were added when a file is loaded. This tracking basically happens by checking which constants were added invoking Object.constants before and after the file was loaded. Tracking these constants take more than 50% of the time in the request, mainly because invoking Object.constants is slow.

In other words, the main reason for an implementation to perform better in the benchmarks showed in the blog post is if it can calculate Object.constants faster. Those results do not mean at all that an implementation is more suitable than other for Rails development. In order to have real results, we would need a real application that is not made of 1000 scaffold (or, in this case, 1000 helpers).

In any case, if the root cause is in loading all helpers, how can we make it better? There are a few things:

1) Obviously, the problem can be fixed by having less helper files. Since Rails scaffold automatically generates helper files, it is common that applications have a bunch of empty helpers. Get rid of them. If you prefer you can even turn off the automatic generation of helpers in scaffold by adding the following to your application configuration:

config.generators.helper = false

2) If you simply don’t want to include all helpers, there is a method called clear_helpers that you could invoke in your ApplicationController. This method won’t fix the problem because it is invoked too late, after all the helpers were already loaded. So you get the feature, but not the performance improvement.

3) Rails master (upcoming Rails 3.1) has a configuration option that allows you to effectively turn these helpers off getting both the feature and the performance improvement:

config.action_controller.include_all_helpers = false

Boot performance

The second blog post shows how Rails boot time performs in different implementations. Since it was not made explicit in which environment those benchmarks were executed, I will assume it happened on development.

At the end of the second blog post, it tries to associate the performance of booting Rails in development with the amount of code inside the app. However, when you boot an application in development, no model, controller or helper is loaded at all unless you explicitly access them in an initializer or in your routes file. Once again, you can check that by adding some puts to your classes.

So, you may ask one more time, what makes booting up so slow?

Rails 3 has a new router that can match paths very fast, but in order to do so, it needs to compile each route into a regular expression and that takes some time (although it could probably be made faster). And it is exactly the routes compilation that is slow on boot time. We can easily reproduce it by adding the following to our router:

Foo::Application.routes.draw do
  1000.times do |index|
    resources :"posts#{index}"
  end
end

This took 55 seconds on my machine using REE which is quite close to the value that he showed on his benchmark.

Again, benchmarking code is important, but more important is to correctly interpret the results. In his example, it is likely that most of Rails booting time is spent on compiling the routes and the benchmark just shows how good different Ruby implementations are in handling all these regular expressions.

Wrapping up

Much more interesting benchmarks for Rails boot time would actually be performed in production environment, which actually has to load all the code inside the app folder and compile the routes file. Regardless, developers starting new applications should always be skeptical about choosing a Ruby implementation based on other application’s benchmarks.

When starting out a new application, any Ruby implementation should suit you just fine unless you have a stronger constraint (like Java integration). Once your application starts to grow and you want to evaluate how well it performs in different implementations, you should do your own benchmarks and see how it goes. In any case, don’t jump into conclusions. If you need to investigate deeper, each implementation has its own sets of benchmarking and profiling tools that may help you really understand what is actually slow and how to improve it.

I also want to thank ruby-prof authors and maintainers, for such an amazing tool, and Yehuda Katz, who helped me profile a demo Rails application in order to write this detailed response.

And you? Have you done benchmarks in your applications and found any interesting data you would like to share?

Note: Devise 1.1.6 broke compatibility with Rails versions prior to 3.0.4, this has been fixed on Devise 1.1.7.

Devise 1.1.6 has just been released and it follows Rails 3.0.4 release. Rails 3.0.4 changes how CSRF works and adds a new method called handle_unverified_request that should be properly overridden by authentication frameworks. Devise 1.1.6 implements this method and others small security fixes.

If you have updated to Rails 3.0.4, you must update to Devise 1.1.6. Those using Devise 1.2.rc should use master for awhile, another RC is coming soon. For more information, check out the CHANGELOG.

Devise 1.0.10 was also released with the same fixes for Rails 2.3.11.