Embracing REST with mind, body and soul

UPDATE: ActionController::Renderer was renamed to ActionController::Responder, so this post was changed to properly reflect such changes.

About two and a half years ago, resources started to be a first class citizen in Rails when version 1.2 was released and it was all about RESTful admiration and HTTP Lovefest. Since then we’ve added map.resources to our routes, started to use different formats in respond_to and really learned how to love all HTTP verbs.

Your application entry point (the router) has become completely RESTful, but it still haven’t reached ActionPack core. Today we are bringing the missing ingredient: make your controllers more resource aware.

The first step: respond_with(@resource)

About one week ago the first step was given. We brought Merb’s provide/display into Rails, just as DHH proposed: you can define supported formats at the class level and tell in the instance the resource to be represented by those formats. Let’s see some code:

  class UsersController < ApplicationController
    respond_to :html, :xml, :json

    def index
      @users = User.all
      respond_with(@users)
    end
  end

It works like this: when a request comes, for example with format xml, it will first search for a template at users/index.xml. If the template is not available, it tries to render the resource given (in this case, @users) by calling :to_xml on it. Before Rails 3.0, the equivalent to the index action above would be:

  class UsersController < ApplicationController
    def index
      @users = User.all
      respond_to do |format|
        format.html
        format.xml { render :xml => @users }
        format.json { render :json => @users }
      end
    end
  end

The gain with respond_with introduction is more obvious if you compare index, new and show actions:

  class UsersController < ApplicationController
    respond_to :html, :xml, :json

    def index
      @users = User.all
      respond_with(@users)
    end

    def new
      @user = User.new
      respond_with(@user)
    end

    def show
      @user = User.find(params[:id])
      respond_with(@user)
    end
  end

With older version:

  class UsersController < ApplicationController
    def index
      @users = User.all
      respond_to do |format|
        format.html
        format.xml { render :xml => @users }
        format.json { render :json => @users }
      end
    end

    def new
      @user = User.new
      respond_to do |format|
        format.html
        format.xml { render :xml => @user }
        format.json { render :json => @user }
      end
    end

    def show
      @user = User.find(params[:id])
      respond_to do |format|
        format.html
        format.xml { render :xml => @user }
        format.json { render :json => @user }
      end
    end
  end

However, even if respond_with is full featured (Ryan Daigle has done an excellent job covering all respond_with features), it started to show some flaws on create, update and destroy actions. A default create action could be written with respond_with as:

  def create
    @user = User.new(params[:user])
    if @user.save
      flash[:notice] = "User was created successfully."
      respond_with(@user, :status => :created, :location => @user) do |format|
        format.html { redirect_to @user }
      end
    else
      respond_with(@user.errors, :status => :unprocessable_entity) do |format|
        format.html { render :action => :new }
      end
    end
  end

You can notice that:

  1. You have to call respond_with twice;
  2. On the first respond_with, you have to give the location twice. One as a hash and other as parameter to redirect_to;
  3. And by giving a block to respond_with, you focus more on the exception than on the default behavior.

Suddenly we realized that respond_with is useful just for GET requests. There was no HTTP Lovefest, it was more like HTTP monotheism.

2. Second step: Love all

At this point, we started to ask ourselves: why can’t respond_with include HTTP verb semantics? Isn’t that what RESTful is all about?

After this commit, we brought all HTTP verbs to respond_with, but only for resourceful formats like xml and json (ie. formats that don’t need to render a template). Then our create action with POST request could be rewritten as:

  def create
    @user = User.new(params[:user])
    respond_with(@user) do |format|
      if @user.save
        flash[:notice] = "User was created successfully."
        format.html { redirect_to @user }
      else
        format.html { render :action => :new }
      end
    end
  end

Internally, when a xml request happened, respond_with would check the current request method (in this case, POST) and whether the resource has errors or not. Depending on these values, it will render the resource or the resource errors, setting accordingly the status and location headers. Now we just have to worry with non-RESTful requests, like html, mobile and iphone… (which we call navigational formats).

Personally, I was quite happy with the results at this iteration, since it solves two of the three problems exposed previously. However, Jeremy Kemper and Yehuda Katz wanted more. And they were right, yeah!

3. Third step: Responder

In step 2, we were able to abstract POST, PUT and DELETE requests for formats like xml and json, but we still would have to repeat html behavior through all controllers, even if almost all of them behave similarly.

So what we want is a simple way to tell the controller how to render our resources depending on the format AND HTTP verb. In this commit, we’ve added ActionController::Responder.

By default, ActionController::Responder holds all formats behavior in a method called to_format. It’s similar to this:

  def to_format
    return render unless resource.respond_to?(:"to_#{format}")

    if get?
      render format => resource
    elsif has_errors?
      render format => resource.errors, :status => :unprocessable_entity
    elsif post?
      render format => resource, :status => :created, :location => resource
    else
      head :ok
    end
  end

As you can see, it renders the resource based on the HTTP verb and whether it has errors or not. If some format, like :html, does not fall into the to_format behavior, we just need to define a to_html in ActionController::Responder, which by default is:

  def to_html
    if get?
      render
    elsif has_errors?
      render :action => (post? ? :new : :edit)
    else
      redirect_to resource
    end
  end

As result, you have your resources representation encapsulated in one place. Your controller code just have to send the resource using respond_with(@resource) and respond_with will call ActionController::Responder which will know what to do. Our create action (POST request) can then be written as:

  def create
    @user = User.new(params[:user])
    flash[:notice] = "User was created successfully." if @user.save
    respond_with(@user)
  end

If you need to change the redirect URL, you can overwrite just the html behavior:

  def create
    @user = User.new(params[:user])
    flash[:notice] = "User was created successfully." if @user.save
    respond_with(@user) do |format|
      format.html { redirect_to user_confirmation_url }
    end
  end

On the other hand, if you want to change the redirect url and the Location header for XML and JSON, you can simply give :location as option:

  def create
    @user = User.new(params[:user])
    flash[:notice] = "User was created successfully." if @user.save
    respond_with(@user, :location => user_confirmation_url)
  end

The best of all is that the responder implementation is quite simple and straight-forward, but still powerful. We haven’t enforced any restriction in the API. Anything that responds to :call can be a responder, so you can create your custom classes or even give procs, fibers and so on.

Embrace REST in your design and enjoy a consistent behavior through all your controllers. Spread the word!

34 responses to “Embracing REST with mind, body and soul”

  1. Yehuda Katz says:

    This is looking really great José. There are still a few changes we (you?) will get in over the next few days:

    * Using the Renderer for default render.
    * Being able to override the renderer used declaratively at the class level (renderer MySpecialRenderer)
    * Inferring the instance variable names to make respond_with the default if performed? is false

  2. Yehuda Katz says:

    This is looking really great José. There are still a few changes we (you?) will get in over the next few days:

    * Using the Renderer for default render.
    * Being able to override the renderer used declaratively at the class level (renderer MySpecialRenderer)
    * Inferring the instance variable names to make respond_with the default if performed? is false

  3. Joe Fiorini says:

    This looks great. Makes the representational part of REST much easier. Might even give resource controller a run for its money! My only question is the title of your article being “Embracing REST with mind, body, and soul”, where’s the “State Transfer” part of REST? In REST as it was originally defined, there should be no need to give out URIs; they should be part of the resource state, therefore allowing one to seamlessly transfer between one resource’s state and another. I haven’t heard much about doing this in Rails, but I’d be interested to see how it could be embedded into the framework. That would make Rails truly innovative.

  4. Joe Fiorini says:

    This looks great. Makes the representational part of REST much easier. Might even give resource controller a run for its money! My only question is the title of your article being “Embracing REST with mind, body, and soul”, where’s the “State Transfer” part of REST? In REST as it was originally defined, there should be no need to give out URIs; they should be part of the resource state, therefore allowing one to seamlessly transfer between one resource’s state and another. I haven’t heard much about doing this in Rails, but I’d be interested to see how it could be embedded into the framework. That would make Rails truly innovative.

  5. José Valim says:

    Joe, good point.

    Do you think that State Transfer is the mind, body or soul of REST? 🙂
    Can you call me on irc (#thor, #rails-contrib or #remarkable), so we can discuss it further?

  6. José Valim says:

    Joe, good point.

    Do you think that State Transfer is the mind, body or soul of REST? 🙂
    Can you call me on irc (#thor, #rails-contrib or #remarkable), so we can discuss it further?

  7. taelor says:

    great post on a preview of whats to come! looks really clean with Renderer now.

    Question, where will this code sit in the directory structure if we want to define our own?

  8. taelor says:

    great post on a preview of whats to come! looks really clean with Renderer now.

    Question, where will this code sit in the directory structure if we want to define our own?

  9. José Valim says:

    Taelor, good question.

    Not yet, there are quite some enhancements we want to do, but it’s better to wait for some feedback before working on new features.

    The best way you can help is trying out Rails 3.0 and tell us how it works for you. I’m usually available on irc, so we can always discuss about it.

    Regards!

  10. José Valim says:

    Taelor, good question.

    Not yet, there are quite some enhancements we want to do, but it’s better to wait for some feedback before working on new features.

    The best way you can help is trying out Rails 3.0 and tell us how it works for you. I’m usually available on irc, so we can always discuss about it.

    Regards!

  11. Vishnu Gopal says:

    Forgive me, but isn’t this a lot of magic? How do you explain this all to new users? More and more I’m preferring an approach like Sinatra—back to simple, humble (understandable) roots.

  12. Vishnu Gopal says:

    Forgive me, but isn’t this a lot of magic? How do you explain this all to new users? More and more I’m preferring an approach like Sinatra—back to simple, humble (understandable) roots.

  13. Adam says:

    José,

    Joe makes an important point.

    Right now, you have to infer the URLs of representations which is explicitly forbidden (see http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven)
    From the above referenced article:

    “A REST API should be entered with no prior knowledge beyond the initial URI (bookmark) and set of standardized media types that are appropriate for the intended audience (i.e., expected to be understood by any client that might use the API). From that point on, all application state transitions must be driven by client selection of server-provided choices that are present in the received representations or implied by the user’s manipulation of those representations. The transitions may be determined (or limited by) the client’s knowledge of media types and resource communication mechanisms, both of which may be improved on-the-fly (e.g., code-on-demand). [Failure here implies that out-of-band information is driving interaction instead of hypertext.]

  14. Adam says:

    José,

    Joe makes an important point.

    Right now, you have to infer the URLs of representations which is explicitly forbidden (see http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven)
    From the above referenced article:

    “A REST API should be entered with no prior knowledge beyond the initial URI (bookmark) and set of standardized media types that are appropriate for the intended audience (i.e., expected to be understood by any client that might use the API). From that point on, all application state transitions must be driven by client selection of server-provided choices that are present in the received representations or implied by the user’s manipulation of those representations. The transitions may be determined (or limited by) the client’s knowledge of media types and resource communication mechanisms, both of which may be improved on-the-fly (e.g., code-on-demand). [Failure here implies that out-of-band information is driving interaction instead of hypertext.]

  15. Tom Ward says:

    I thought the respond_with work was good, but together with all these changes REST support is getting better. The proliferation of restful plugins are now slowly becoming less and less useful. Great work.

  16. Tom Ward says:

    I thought the respond_with work was good, but together with all these changes REST support is getting better. The proliferation of restful plugins are now slowly becoming less and less useful. Great work.

  17. José Valim says:

    @Vishnu I have to disagree at this point that this is a lot of magic. Yes, there are a lot of things happening, but the code is quite straight-forward. respond_with just invokes the ActionController::Renderer class which is less than 100LOC.

    Try to consider a huge application, where all those controllers have this common code for scaffold actions (index, show, create, update…). If you want to start using http cache, you have to do that at only one place. This is about treating your resources equally and bring consistency to your controllers.

    Besides Rails 3.0 is also bringing to the table modularity. If you want, you can get a Rails AbstractController and implement a SinatraComapatibility layer on top of it. You don’t need to use any of the additional stuff.

  18. José Valim says:

    @Vishnu I have to disagree at this point that this is a lot of magic. Yes, there are a lot of things happening, but the code is quite straight-forward. respond_with just invokes the ActionController::Renderer class which is less than 100LOC.

    Try to consider a huge application, where all those controllers have this common code for scaffold actions (index, show, create, update…). If you want to start using http cache, you have to do that at only one place. This is about treating your resources equally and bring consistency to your controllers.

    Besides Rails 3.0 is also bringing to the table modularity. If you want, you can get a Rails AbstractController and implement a SinatraComapatibility layer on top of it. You don’t need to use any of the additional stuff.

  19. José Valim says:

    Adam and Joe,

    I was able to discuss that with some Rails Core guys and there are definitely some interest in adding this to Rails. I was also able to find a couple plugins in Github that does something similar. It definitely would close the deal.

  20. José Valim says:

    Adam and Joe,

    I was able to discuss that with some Rails Core guys and there are definitely some interest in adding this to Rails. I was also able to find a couple plugins in Github that does something similar. It definitely would close the deal.

  21. Yehuda Katz says:

    @vishnu “Forgive me, but isn’t this a lot of magic? How do you explain this all to new users? More and more I’m preferring an approach like Sinatra—back to simple, humble (understandable) roots”

    For simple apps, you should use a simple approach. Rails still supports the simple approach (even without respond_to). You could even do self.response_body = “Whatever”. However, in more complex apps, modeling REST behavior, especially when it’s almost always identical, can get very difficult.

    This provides us with a way to wrap up a default set of REST behavior, and more importantly, allow you to define the semantics of your objects if you don’t want to follow our conventions. For most users, respond_with will have the pleasant effect of reducing code in their controllers (in non-confusing ways for the simple cases). Users with more complex apps will appreciate not having to stuff all logic in a single object, and the ability to easily define presentation logic that depends on a specific resource + the current request.

  22. Yehuda Katz says:

    @vishnu “Forgive me, but isn’t this a lot of magic? How do you explain this all to new users? More and more I’m preferring an approach like Sinatra—back to simple, humble (understandable) roots”

    For simple apps, you should use a simple approach. Rails still supports the simple approach (even without respond_to). You could even do self.response_body = “Whatever”. However, in more complex apps, modeling REST behavior, especially when it’s almost always identical, can get very difficult.

    This provides us with a way to wrap up a default set of REST behavior, and more importantly, allow you to define the semantics of your objects if you don’t want to follow our conventions. For most users, respond_with will have the pleasant effect of reducing code in their controllers (in non-confusing ways for the simple cases). Users with more complex apps will appreciate not having to stuff all logic in a single object, and the ability to easily define presentation logic that depends on a specific resource + the current request.

  23. […] Embracing REST with mind, body and soul – Jose Valim writes about some of the changes coming in Rails 3. […]

  24. Gaizka says:

    Great post!

    I have a question about your last example, when you want to overwrite the JSON format behaviour.

    I think that in your code, a new User *is* begin created (even if using JSON), but at the end you respond with “You cannot create an user through JSON API”.

    How could you rewrite that example wihout looking too clumsy?

    Thanks!!

  25. Gaizka says:

    Great post!

    I have a question about your last example, when you want to overwrite the JSON format behaviour.

    I think that in your code, a new User *is* begin created (even if using JSON), but at the end you respond with “You cannot create an user through JSON API”.

    How could you rewrite that example wihout looking too clumsy?

    Thanks!!

  26. José Valim says:

    Gaizka, good point. Already fixed! Thanks!

  27. José Valim says:

    Gaizka, good point. Already fixed! Thanks!

  28. Jesse Rankin says:

    +1 for adding the state transfer functionality. That is something that has always bugged me about rails’ rest implementation.

  29. Jesse Rankin says:

    +1 for adding the state transfer functionality. That is something that has always bugged me about rails’ rest implementation.

  30. […] Embracing REST with mind, body and soul …and more on Rails 3 […]

  31. […] Embracing REST with mind, body and soul | Plataforma Blog (tags: rails ruby rest restful programming) […]

  32. […] minha primeira feature após os geradores de código foi o ActionController::Responder e atualmente estou trabalhando na otimização de callbacks e observers do ActiveRecord (assim como […]

  33. […] Resources. This initial implementation provided a nice test suite and several use cases for a improved Rails’ implementation, based in Responders, which encapsulates all the behavior in one class, and can be added, modified […]