Posts tagged "scopes"

Here at Plataformatec we use Github Pull Requests a lot for code review and this usually yields tons of constructive comments and excellent discussions from time to time. One of the recent topics was about whether we should use scopes or class methods throughout the project to be consistent. It’s also not hard to find discussions about it all over the internet. The classic comment usually boils down to “there is no difference between them” or “it is a matter of taste”. I tend to agree with both sentences, but I’d like to show some slight differences that exist between both.

Defining a scope

First of all, lets get a better understanding about how scopes are used. In Rails 3 you can define a scope in two ways:

class Post < ActiveRecord::Base
  scope :published, where(status: 'published')
  scope :draft, -> { where(status: 'draft') } 
end

The main difference between both usages is that the :published condition is evaluated when the class is first loaded, whereas the :draft one is lazy evaluated when it is called. Because of that, in Rails 4 the first way is going to be deprecated which means you will always need to declare scopes with a callable object as argument. This is to avoid issues when trying to declare a scope with some sort of Time argument:

class Post < ActiveRecord::Base
  scope :published_last_week, where('published_at >= ?', 1.week.ago)
end

Because this won’t work as expected: 1.week.ago will be evaluated when the class is loaded, not every time the scope is called.

Scopes are just class methods

Internally Active Record converts a scope into a class method. Conceptually, its simplified implementation in Rails master looks something like this:

def self.scope(name, body)
  singleton_class.send(:define_method, name, &body)
end

Which ends up as a class method with the given name and body, like this:

def self.published
  where(status: 'published')
end

And I think that’s why most people think: “Why should I use a scope if it is just syntax sugar for a class method?”. So here are some interesting examples for you to think about.

Scopes are always chainable

Lets use the following scenario: users will be able to filter posts by statuses, ordering by most recent updated ones. Simple enough, lets write scopes for that:

class Post < ActiveRecord::Base
  scope :by_status, -> status { where(status: status) }
  scope :recent, -> { order("posts.updated_at DESC") }
end

And we can call them freely like this:

Post.by_status('published').recent
# SELECT "posts".* FROM "posts" WHERE "posts"."status" = 'published' 
#   ORDER BY posts.updated_at DESC

Or with a user provided param:

Post.by_status(params[:status]).recent
# SELECT "posts".* FROM "posts" WHERE "posts"."status" = 'published' 
#   ORDER BY posts.updated_at DESC

So far, so good. Now lets move them to class methods, just for the sake of comparing:

class Post < ActiveRecord::Base
  def self.by_status(status)
    where(status: status)
  end
 
  def self.recent
    order("posts.updated_at DESC")
  end
end

Besides using a few extra lines, no big improvements. But now what happens if the :status parameter is nil or blank?

Post.by_status(nil).recent
# SELECT "posts".* FROM "posts" WHERE "posts"."status" IS NULL 
#   ORDER BY posts.updated_at DESC
 
Post.by_status('').recent
# SELECT "posts".* FROM "posts" WHERE "posts"."status" = '' 
#   ORDER BY posts.updated_at DESC

Oooops, I don’t think we wanted to allow these queries, did we? With scopes, we can easily fix that by adding a presence condition to our scope:

scope :by_status, -> status { where(status: status) if status.present? }

There we go:

Post.by_status(nil).recent
# SELECT "posts".* FROM "posts" ORDER BY posts.updated_at DESC
 
Post.by_status('').recent
# SELECT "posts".* FROM "posts" ORDER BY posts.updated_at DESC

Awesome. Now lets try to do the same with our beloved class method:

class Post < ActiveRecord::Base
  def self.by_status(status)
    where(status: status) if status.present?
  end
end

Running this:

Post.by_status('').recent
NoMethodError: undefined method `recent' for nil:NilClass

And :bomb:. The difference is that a scope will always return a relation, whereas our simple class method implementation will not. The class method should look like this instead:

def self.by_status(status)
  if status.present?
    where(status: status)
  else
    all
  end
end

Notice that I’m returning all for the nil/blank case, which in Rails 4 returns a relation (it previously returned the Array of items from the database). In Rails 3.2.x, you should use scoped there instead. And there we go:

Post.by_status('').recent
# SELECT "posts".* FROM "posts" ORDER BY posts.updated_at DESC

So the advice here is: never return nil from a class method that should work like a scope, otherwise you’re breaking the chainability condition implied by scopes, that always return a relation.

Scopes are extensible

Lets get pagination as our next example and I’m going to use the kaminari gem as basis. The most important thing you need to do when paginating a collection is to tell which page you want to fetch:

Post.page(2)

After doing that you might want to say how many records per page you want:

Post.page(2).per(15)

And you may to know the total number of pages, or whether you are in the first or last page:

posts = Post.page(2)
posts.total_pages # => 2
posts.first_page? # => false
posts.last_page?  # => true

This all makes sense when we call things in this order, but it doesn’t make any sense to call these methods in a collection that is not paginated, does it? When you write scopes, you can add specific extensions that will only be available in your object if that scope is called. In case of kaminari, it only adds the page scope to your Active Record models, and relies on the scope extensions feature to add all other functionality when page is called. Conceptually, the code would look like this:

scope :page, -> num { # some limit + offset logic here for pagination } do
  def per(num)
    # more logic here
  end
 
  def total_pages
    # some more here
  end
 
  def first_page?
    # and a bit more
  end
 
  def last_page?
    # and so on
  end
end

Scope extensions is a powerful and flexible technique to have in our toolchain. But of course, we can always go wild and get all that with class methods too:

def self.page(num)
  scope = # some limit + offset logic here for pagination
  scope.extend PaginationExtensions
  scope
end
 
module PaginationExtensions
  def per(num)
    # more logic here
  end
 
  def total_pages
    # some more here
  end
 
  def first_page?
    # and a bit more
  end
 
  def last_page?
    # and so on
  end
end

It is a bit more verbose than using a scope, but it yields the same results. And the advice here is: pick what works better for you but make sure you know what the framework provides before reinventing the wheel.

Wrapping up

I personally tend to use scopes when the logic is very small, for simple where/order clauses, and class methods when it involves a bit more complexity, but whether it receives an argument or not doesn’t really matter much to me. I also tend to rely more on scopes when doing extensions like showed here, since it’s a feature that Active Record already gives us for free.

I think it’s important to clarify the main differences between scopes and class methods, so that you can pick the right tool for the job™, or the tool that makes you more comfortable. Whether you use one or another, I don’t think it really matters, as long as you write them clear and consistently throughout your application.

Do you have any thought about using scopes vs class methods? Make sure to leave a comment below telling us what you think, we’d love to hear.

You probably know that Active Record got a facelift and is now powered by Active Relation. A new chainable-award-winning-lazy API was added and received great feedback! However, as more and more people are trying Rails 3 beta, a small incompatibility between the old and new syntax was found. This post explains this incompatibility and how it was solved.

The issue

Quoting the Lighthouse ticket, imagine the following scenario in Rails 2.3:

class Page < ActiveRecord::Base
  default_scope :conditions => { :deleted_at => nil }
 
  def self.deleted
    with_exclusive_scope :find => { :conditions => "pages.deleted_at IS NOT NULL" } do
      all
    end
  end
end

If you rewrite it to the new 3.0 syntax, your first attempt would probably be:

class Page < ActiveRecord::Base
  default_scope where(:deleted_at => nil)
 
  def self.deleted
    with_exclusive_scope :find => where('pages.deleted_at IS NOT NULL') do
      all
    end
  end
end

However, if you try it out on console, you will find out it does not work as expected:

Page.all         #=> SELECT "pages".* FROM "pages" WHERE ("pages"."deleted_at" IS NULL)
Page.deleted.all #=> SELECT "pages".* FROM "pages" WHERE ("pages"."deleted_at" IS NULL) AND ("pages"."deleted_at" IS NOT NULL)

To understand why it does not work, let’s take a look at the source code!

Investigating the issue

With Active Relation, Active Record is no longer responsible to build queries. That said, ActiveRecord::Base is not the one that implements where() and friends, in fact, it simply delegates to an ActiveRecord::Relation object. From ActiveRecord::Base source code:

delegate :select, :group, :order, :limit, :joins, :where, :preload, :eager_load, :includes, :from, :lock, :readonly, :having, :create_with, :to => :scoped

And the scoped implementation is shown below:

def scoped(options = nil)
  if options.present?
    scoped.apply_finder_options(options)
  else
    current_scoped_methods ? relation.merge(current_scoped_methods) : relation.clone
  end
end
 
def relation
  @relation ||= ActiveRecord::Relation.new(self, arel_table)
  finder_needs_type_condition? ? @relation.where(type_condition) : @relation
end

As you can see, scoped always returns an ActiveRecord::Relation that you build your query on top of (notice that ARel::Relation is not the same as ActiveRecord::Relation).

Besides, if there is any current_scoped_methods, the scoped method is responsible to merge this current scope into the raw relation. This is where things get interesting.

When you create your model, current_scoped_methods returns by default nil. However, when you define a default_scope, the current scope now becomes the relation given to default_scope, meaning that, every time you call scoped, it returns the raw relation merged with your default scope.

The whole idea of with_exclusive_scope is to be able to make a query without taking the default scope into account, just the relation you give in as argument. That said, it basically sets the current_scope_methods back to nil, so every time you call scoped to build your queries, it will be built on top of the raw relation without the default scope.

With that in mind, if we look again at the code which we were trying to port from Rails 2.3, we can finally understand what was happening:

def self.deleted
  with_exclusive_scope :find => where('pages.deleted_at IS NOT NULL') do
    self
  end
end

When we called where('pages.deleted_at IS NOT NULL') above, we were doing the same as: scoped.where('pages.deleted_at IS NOT NULL'). But, as scoped was called outside the with_exclusive_scope block, it means that the relation given as argument to :find was built on top of default_scope explaining the query we saw as results.

For example, the following syntax would work as expected:

def self.deleted
  with_exclusive_scope do
    where('pages.deleted_at IS NOT NULL').all
  end
end

Since we are calling where inside the block, the scoped method no longer takes the default scope into account. However, moving the relation inside the block is not the same as specifying it to :find, because if we were doing three queries inside the block, we would have to specify the same relation three times (or refactor the whole code to always do a query on top of this new relation).

That said, it seems the previous with_exclusive_scope syntax does not suit very well with ActiveRecord’s new API. Maybe is it time for change? Can we provide a better API? Which are the use cases?

Identifying the use cases

The with_exclusive_scope method has mainly two use cases. The first one, which we just discussed above, is to allow us to make a query without taking the default scope into account inside our models:

def self.deleted
  with_exclusive_scope do
    where('pages.deleted_at IS NOT NULL').all
  end
end

While this code looks ok, if we think about relations, we will realize that we don’t need to give a block to achieve the behavior we want. If the scoped method returns a raw relation with the default scope, couldn’t we have a method that always returns the raw relation? Allowing us to build our query without taking the default scope into account?

In fact, this method was already implemented in Active Record and it is called unscoped. That said, the code above could simply be rewritten as:

def self.deleted
  unscoped.where('pages.deleted_at IS NOT NULL').all
end

Much simpler! So, it seems that we don’t need to support the block usage at all, confirm?

Deny! Going back to the Page example above, it seems we should never see deleted pages, that’s why we set the default_scope to :deleted_at => nil. However, if this application has an admin section, the admin may want to see all pages, including the deleted ones.

That said, what we could do is to have one controller for the normal User and another for the Admin. In the former, we would always use Page.all, and Page.unscoped.all in the latter.

However, if these controllers and views are very similar, you may not want do duplicate everything. Maybe it would be easier if we do something like this:

def resource_class
  if current_user.is_admin?
    Page.unscoped
  else
    Page
  end
end

And, instead of always referencing the Page class directly in our actions, we could call resource_class. While this solution is also ok, there is a final alternative, that would require no changes to the current code. If you want to use the same controller for different roles, but changing the scope of what they are allowed to see, you could simply use an around_filter to change the model scope during the execution of an action. Here is an example:

class PagesController < ApplicationController
  around_filter :apply_scope
 
  # some code ...
 
  protected
 
  def apply_scope
    if current_user.admin?
      Page.with_exclusive_scope { yield }
    else
      yield
    end
  end
end

That said, being allowed to give a block to with_exclusive_scope is actually useful and since we want to deprecate with_exclusive_scope in favor of unscoped in the future, we brought this very same syntax to unscoped as well:

def apply_scope
  if current_user.admin?
    Page.unscoped { yield }
  else
    yield
  end
end

Tidying it up

Well, after the behavior in with_exclusive_scope was properly ported to the new API, we need to be sure we are not forgetting about anything… wait, actually we are.

with_exclusive_scope has an evil twin brother called with_scope which behaves very similarly, except that it always build the query on top of the scoped relation. It works like this:

class Page < ActiveRecord::Base
  default_scope where(:deleted_at => nil)
end
 
Page.with_scope :find => { :conditions => { :active => true } } do
  Page.all #=> Bring all active pages that were not deleted
end

However, this feels way too hash-ish. Of course, we could use relations to make it a bit prettier:

Page.with_scope :find => where(:active => true) do
  Page.all #=> Bring all active pages that were not deleted
end

This is ok, but it seems that we could improve it even more. That said, we added a new method to relations, called scoping:

Page.where(:active => true).scoping do
  Page.all #=> Bring all active pages that were not deleted
end

Yeah! Sign me up ’cause this looks way better than the previous syntax! And, if you check the original commit, you will notice the unscoped method with a block simply delegates scoping:

def unscoped
  block_given? ? relation.scoping { yield } : relation
end

So, with unscoped and scoping implemented, we just need to commit, git push and be happy, confirm? Deny! There is one last case to check.

create_with

If you payed attention properly, you can notice that every time we called with_exclusive_scope and with_scope, we always passed { :find => relation } as hash, instead of simply giving the relation. This happens because these methods accept two hash keys: find and create.

As you may expect, one specifies the behavior for create and the other for finding. In most of the cases, they are exactly the same and work with the new syntax:

page = Page.where(:active => true).new
page.active #=> true

However, for obvious reasons, this only works if the conditions are given as a hash. Consider this case:

page = Page.where("active = true").new
page.active #=> nil

That said, there may be a few scenarios where you want to specify the creation conditions on its own, explaining the :find and :create options in with_exclusive_scope and with_scope methods. So, how can I achieve it with the new syntax? Easy!

page = Page.create_with(:active => true).new
page.active #=> true

If you provide both conditions as a hash and create_with, create_with always have higher priority:

page = Page.where(:active => false).create_with(:active => true).new
page.active #=> true

Note this syntax already existed, we are just making it explicit now as part of the new API! That said, commit, push and be happy!

Wrapping up

All in all, with_exclusive_scope and with_scope are now part of the old ActiveRecord API giving place to the new, strong and vibrant unscoped and scoping methods!

However, they are not going to be deprecated now. They will follow the same deprecation strategy as all the current methods.

And you? What do you think about this new scoping API?