Posts tagged "rails"

Or, even better, why your web framework should not adopt a CGI-based API.

For the past few years I have been studying and observing the development of different emerging languages closely with a special focus on web frameworks/servers. Unfortunately, most of the new web frameworks are following the Rack/WSGI specification which may be a mistake depending on the platform you are targeting (particularly true for Erlang and Node.js which have very strong streaming foundations and is by default part of their stack).

This blog post is an attempt to detail the limitations in the Rack/CGI-based APIs that the Rails Core Team has found while working with the streaming feature that shipped with Rails 3.1 and why we need better abstractions in the long term.

Case in study

The use case we have in mind here is streaming. In Rails, we have focused on streaming as a way to quickly return the head of the HTML page to the browser, so the browser can start downloading assets (like javascript and stylesheet) while the server is generating the rest of the page. There is a great entry on Rails weblog about streaming in general and a Railscast if you want to focus on how to use it in your Rails applications. However, streaming is not only limited to HTML responses and can also be useful in API endpoints, for example, to stream search results as they pop-up or to synchronize with mobile devices.

The Rack specification in a nutshell

In Rack, the response is made by an array with three elements: [status, headers, body]

The body can be any object that responds to the method each. This means streaming can be done by passing an object that will, for example, lazily read a file and stream chunks when each is called.

A Rack application is any object that implements the method call and receives an environment hash/dictionary with the request information. When I said above that most new web frameworks are following the Rack specification, is because they are introducing an API similar to this one just described.

The issue

In order to understand the issue, we will consider three entities: the client, the server and the application. The client (for example a browser) sends a request to the server which forwards it to an application. In this case, the server and the application are communicating via the Rack API.

The issue in streaming cases is that, sending a response back from the application does not mean the application finished processing. For example, consider a middleware (a middleware is an object that sits in between the server and our application) that opens up a connection to the database for the duration of the request and cleans it afterwards:

def call(env)
  connection = DB.checkout_connection
  env["db.connection"] = connection
  DB.checkin_connection connection

Without streaming, it would work as follow:

  1. The server receives a request and passes it down the stack
  2. The request reaches the middleware
  3. The middleware checks out the connection
  4. The application is invoked, renders a view accessing the database using the connection and returns the rendered view as a string
  5. The middleware checks the connection back in
  6. The response is sent back to the client

With streaming, this would happen:

  1. The server receives a request and passes it down the stack
  2. The request reaches the middleware
  3. The middleware checks out the connection
  4. The app is called but does not render anything. Instead it returns a lazy object as response body that will stream the HTML page in chunks as the `each` method is called
  5. The middleware checks the connection back in
  6. Back in the server, we have received the lazy body and will start to stream it
  7. While streaming the body, since the body is lazily calculated, now is the time it must access the database. But, since the middleware already checked the connection back in, our code will fail with a “not connected” exception

The first reaction to solve this issue is to ensure that all streaming happens inside the application, i.e. the application would have a mechanism to stream the response back and only when it is done it would return the Rack response back. However, if the application does this, any middleware that desires to modify the header or the response body won’t be able to do so because the response was already streamed from inside the application.

Our work-around for Rails was to create proxies that wrap the response body:

def call(env)
  connection = DB.checkout_connection
  env["db.connection"] = connection
  response = do
    DB.checkin_connection connection

However, this is inefficient and extremely limited (not all middleware can be converted to such approach). In order for streaming to be successful, the underlying server API needs to acknowledge that the headers and the response body can be sent at different times. Not only that, it needs to provide proper callbacks around the response lifecycle (before sending headers, when the response is closed, on each stream, etc).

The trade-off here is that this can no longer be achieved with an easy API as Rack’s. In general, we would like to have a response objects that provides several life-cycle hooks. For example, the middleware above could be rewritten as:

def call(request, response)
  connection = DB.checkout_connection
  request.env["db.connection"] = connection
  response.on_close { DB.checkin_connection(connection) }, response)

The Java Servlet specification is a good example of how request and response objects could be designed to provide such hooks.

Other middleware

In the example above I focused on the database connection middleware but this limitation exists, in one way or the other, in the majority of middleware in a stack. For example, a middleware that rescues any exception inside the application to render a 500 page also needs to be adapted. Other middleware simply won’t work. For instance, Rails ships with a middleware that provides an ETag header based on the body which has to be disabled when streaming.

Looking back

Does this mean moving to Rack was a mistake? Not at all. Rack appeared when the web development Ruby community was fragmented and the simplicity of the Rack API made it possible to unify the different web frameworks and web servers available. Looking back, I would take the standardization provided by Rack any day regardless of the limitations it brings. Now that we have a standard, we are already working on addressing such issues, which leads us to…

Looking forward

Streaming will become more and more important. While working with HTML streaming requires special attention both technically and also in terms of usability, as outlined in Rails’ documentation, API endpoints could benefit from it with basically no extra cost. Not only that, features in HTML5 like server-sent events could easily be built on top of streaming without requiring a specific end-point in your application to handle them.

While CGI was originally streaming friendly, the abstractions we built on top of it (like middleware) are not. I believe web frameworks should be moving towards better connection/socket abstractions and away from the old CGI-based APIs, which served us well but it is finally time for us to let it go.

PS: Thanks to Aaron Patterson (who has also written about this issue in his blog), Yehuda Katz, Konstantin Haase and James Tucker for early review and feedback.


This section was added after the blog post was released based on some common questions.

Q: Isn’t it a bad idea to mix both streaming and non-streaming behavior in the same stack?

That depends on the stack. This is definitely not an issue with Erlang and Node.js since both stacks are streaming based. In Ruby, I believe a threaded jRuby or Thin will allow you to get away with keeping a socket open waiting for responses, but it will probably turn out to be a bad idea with other servers since the process holding the socket won’t be able to respond to any other request.

Q: Is there a need to do everything streaming based when a request/response would be fine?

No, there is no need. The point of the blog post is not to advocate for streaming only frameworks, but simply state that a Rack API may severely limit your streaming capability in case your platform supports it. Personally, I would like to be able to choose and mix both, if my stack allows me to do so.

When David Chelimsky was visiting São Paulo in last April, we invited him to go out for some coffee, beers and brazilian appetizers. We had a great time and we talked about different topics like OO, programming languages, authoring books and, as expected, about testing.

One of the topics in our testing discussion was the current confusion in rspec-rails request specs when using Capybara. There is an open issue for this in rspec-rails’ issues tracker and discussing it personally allowed us to talk about some possible solutions, which could take months in internet time. :)

rspec-rails is a gem that wraps Rails testing behaviors into RSpec’s example groups. For example, the controller example group is based on ActionController::TestCase::Behavior. There are also example groups for views, helpers and so forth, but for now we are interested in the request example group, which is as a wrapper for ActionDispatch::Integration::Runner. The Rails’ integration runner is built on top of rack-test, a great small gem that adds support to methods like get, post, put and delete and handle the rack request and response objects.

This setup with the request example group running on top of Rails’ Integration Runner works fine until you add Capybara to your application (which is always a good idea). The issue is that Capybara by default includes its DSL in the same request example group and that’s when the confusion starts.

Capybara, being an acceptance test framework, does not expose low-level details like a request or response object. In order to access a web page using Capybara, the developer needs to use the method visit (instead of get). To read the accessed page body, the developer must use page instead of manipulating the response.

However, since both Capybara DSL and Rails’ Integration Runner are included in the same example group, both methods visit and get are available! Not only that, even if I visit a web page using Capybara’s visit, I can still access the request and response object that comes from Rails, except that they will be blank since Capybara uses a completely different stack to access the application.

This confusion not only happens inside each test but it also leads to a poor testing suite. I have seen many, many files inside spec/requests that mixes both syntaxes.

Talking to David, I have expressed a possible solution to this problem based on how we have been building applications at Plataformatec. First of all, we start by having two directories: spec/requests and spec/acceptance. Since both are supported by Capybara, this (mostly) works out of the box.

Everything you want to test from the user/browser perspective goes under spec/acceptance. So if you want to test that by filling the body and the title fields and pressing the button “Publish” publishes a new blog post, you will test that under acceptance (protip: we usually have subdirectories inside spec/acceptance based on the application roles, like spec/acceptance/guest, spec/acceptance/admin, etc).

Everything under spec/requests applies to the inner working of your application. Is it returning the proper http headers? Is this route streaming the correct JSON response? Also, since APIs are not part of the user/browser perspective, they are also tested under spec/requests and not under spec/acceptance.

This separation of concerns already helps solving the confusion above. Under spec/acceptance, you should use only Capybara helpers. Inside spec/requests, you are using Rails provided tools. However, this does not solve the underlying problem that both helpers are still included in spec/requests.

Therefore, while this blog post means to provide some guidance for those that run into such problems, we also would like to propose a solution that we discussed with David. The solution goes like this:

1) We change RSpec to no longer generate spec/requests, but both spec/api and spec/features (I have proposed spec/acceptance but David pointed out those are not strictly speaking acceptance tests). The Capybara DSL (visit, page and friends) should not be included in spec/api under any circumstance.

2) We change Capybara to include by default its DSL and RSpec matchers under spec/features and change the feature method to rely on the type :features instead of :requests.

The proposal suggests the addition of two new directories instead of changing the behavior of existing ones in order to be backwards compatible while ensuring a safe and more semantic future for everyone else. David asked me to outline our conversation in a blog post, so we can get some awareness and feedback before undergoing such changes. So, what do you think?

After 4 months of our last major version release, we’re releasing Devise 2.1.0, which includes several bug fixes, some new features and the removal of features deprecated on Devise 2.0. If you’re eager to do the update, please check Devise’s wiki page about Upgrading from 2.0 to 2.1. You can also check the changelog or the commits between 2.0.4 and 2.1.0.

Encrytable is now a gem

As it was only used for old encryption algorithms like sha1 or md5, we have extracted encryptable module to a separated ruby gem. So, if you’re using the encryptable module, you should only require it on your Gemfile and you’re good to go!

Check fields

To allow a developer to cherry-pick which features they want to add to their models, Devise splits its behaviors into modules. One of the consequences of such splitting is that you don’t know if the persistence layer provides all fields required by the behavior. For example, the database authenticatable module requires a encrypted_password field. If the field does not exist, you will end up getting an error during a request. Usually those fields are automatically added to the migration when you call the devise generator, but if you decide to include a module after, you can easily forget to add the new fields.

That said, in order to provide faster feedback, Devise now has a method that checks if a given class is missing any of the required fields and raises an error if so. You can call this method as follow (in case your Devise class is User):


We’ve implemented this feature in an agnostic way, to not depend on a specific ORM, but only to verify if the instance responds to the required fields. So even if your ORM does everything through method_missing, you should be able to use this method (we’re relying that you also have implemented a working respond_to?, which is strongly recommendeded when using method_missing).

When check_fields! is called, Devise will detect the included modules in the given class and, if there is a missing attribute, it will raise a Devise::Models::MissingAttribute exception with a message telling you all required fields that doesn’t exist. You can easily use that method with your favorite test framework:


And then you will be able to check if your migrations have added the correct fields to your database.

A message to module maintainers

If you’re a maintainer of a Devise module, you should add a method to each of your modules called self.required_fields(klass) that returns an array of required fields. If the method is absent, you will get a deprecation warning.

UPDATE: Fixed a class name and corrected a grammar error.

Today I want to show you a project I’ve started over a year ago, during Mendicant University core skills course. For those who don’t know, Mendicant University is a group of skilled software developers that offer courses, mentoring, and help out the community, started by Gregory Brown, and that nowadays counts with some other awesome folks as part of the staff. I highly recommend taking a look at and enrolling.

Back to I18n, during Mendicant University we were supposed to create a project in Ruby, not specifically with Rails, and I decided to scratch my own itch by trying to solve a problem we usually have in Brazil: receiving date/time/numeric input from user interface. I know and have already used the delocalized gem, and it works quite nice, but sometimes I felt a bit uncomfortable about how it handled some parts of localization/parsing. This is mainly due to the need to monkey patch both ActiveRecord to handle input, and ActionView to handle output. Besides that, and most important, I had to come up with some project and I thought that’d be a good challenge :D.

The main goal of this project is to provide a proxy object to use with your ORM (currently ActiveRecord only) that will be responsible for localizing and parsing the date/time/numeric attributes when getting or setting their values, respectively. Lets see some quick examples:

# Include the proxy in your model
class Product < ActiveRecord::Base
  include I18n::Alchemy
# Grab your object from the database
@product   = Product.first
# Instantiate the localized proxy
@localized = @product.localized

Now that we have a localized proxy for the @product object, we can get/set numeric attributes with localized values, such as:

@localized.price = "1.99"
@product.price   # => 1.99
@localized.price # => "1.99"
I18n.with_locale :pt do
  @localized.price = "1,88"
  @product.price   # => 1.88
  @localized.price # => "1,88"

And also date/time attributes, for instance:

@localized.released_at = "12/31/2011"
@product.released_at   # =>, 12, 31)
@localized.released_at # => "12/31/2011"
I18n.with_locale :pt do
  @localized.released_at = "31/12/2011"
  @product.released_at   # =>, 12, 31)
  @localized.released_at # => "31/12/2011"

I18n Alchemy can also receive a hash of attributes, the same way you use with your models when calling new. That means you can use it like this:

# You could be using params[:product] for instance.
I18n.with_locale :pt do
  @localized = @product.localized(:price => "1,88") 
  @product.price   # => 1.88
  @localized.price # => "1,88"

The parsing/localization formats are basically the same ones you already use in your Rails application. You can check the basic locale configuration for I18n Alchemy in its README on github.

Wrapping up

I18n Alchemy is a small and new project which solves most of the problems we commonly face when dealing with localization and parsing of date/time/numeric values. It is tested with Rails 3.0, 3.1 and 3.2 and works with all the basic methods, such as attributes=, assign_attributes, update_attributes and nested attributes as well.

It was a really fun time creating it during Mendicant University, and it took a long time until I decided to release it as a gem. There is still a bunch of things to do, but I wanted to ask you to give it a try and let me know about any feedback you have.

As a side note, if you are interested in knowing more about the design decisions that led this project, you may want to take a look at Gregory Brown’s post on Ruby Best Practices, entitled “Issue #23: SOLID Design Principles”, more specifically in the Open/closed principle topic.

I’m releasing the first 0.0.1 version today, and I hope you find it useful. Have any comments? Let us know!

Rails 4.0 – current master branch at the time of this writing – has recently got a small – yet very useful – addition: ActiveModel::Model. The implementation is really simple, as you can see below:

module ActiveModel
  module Model
    def self.included(base)
      base.class_eval do
        extend  ActiveModel::Naming
        extend  ActiveModel::Translation
        include ActiveModel::Validations
        include ActiveModel::Conversion
    def initialize(params={})
      params.each do |attr, value|
        self.public_send("#{attr}=", value)
      end if params
    def persisted?

Quite straightforward, huh? But what does it do, and what are we supposed to do with it?

ActiveModel::Model: Basic Model implementation

According to the docs, ActiveModel::Model includes all the required interface for an object to interact with ActionPack, using different ActiveModel modules. It includes model name instrospection, conversions, translations and validations. In addition to that, it allows you to initialize the object with a hash of attributes, pretty much like ActiveRecord does.

Wait, what? In short: you can easily extend ActiveModel::Model in a normal Ruby class and use instances of that class with helpers like form_for, dom_id / dom_class, and any other ActionView helper, as you do with ActiveRecord objects. It also gives you known method helpers such as human_attribute_name.

A minimal implementation could be:

class Person
  include ActiveModel::Model
  attr_accessor :name, :age
  validates_presence_of :name
person = => 'bob', :age => '18') # => 'bob'
person.age # => 18
person.valid? # => true

This is really handy, considering that before this addition, we’d have to add all that code to have a model up and running to use with ActionView's form_for, for instance. Ok, it is not that much code to add, but now we don’t even need to remember which modules are required for such integration. And I have to add that I’ve been creating similar classes in different applications lately. Take a moment to think about a contact form, that does not need to be tied to a database: it’s a common scenario to implement using ActiveModel::Model.

Extending Basic Model even more

Note that, by default, ActiveModel::Model implements persisted? to return false, which is the most common case. For instance, when used with form_for, this means that the generated url would post to the create action. You may want to override it in your class to simulate a different scenario:

class Person
  include ActiveModel::Model
  attr_accessor :id, :name
  def persisted? == 1
person = => 1, :name => 'bob')
person.persisted? # => true

Besides that, if for some reason you need to run code on initialize, make sure you call super if you want the attributes hash initialization to happen.

class Person
  include ActiveModel::Model
  attr_accessor :id, :name, :omg
  def initialize(attributes)
    @omg ||= true
person = => 1, :name => 'bob')
person.omg # => true

And remember that, at the end, this is all Ruby: you can include any other module of your own and other ActiveModel modules easily in your class. For instance, lets add callbacks to our model to mimic ActiveRecord's save functionality:

class Person
  include ActiveModel::Model
  extend ActiveModel::Callbacks
  define_model_callbacks :save
  attr_accessor :id, :name
  # Just check validity, and if so, trigger callbacks.
  def save
    if valid?
      run_callbacks(:save) { true }

This gives you before_save, after_save and around_save callbacks. Quick and easy, huh?

Wrapping up

ActiveModel::Model is a really small, handy addition to Rails 4.0, which helps us to get classes that act more like ActiveRecord and easily integrate with ActionPack.

For more detailed information on other features available, please refer to the specific modules included in ActiveModel::Model. Each module includes plenty of docs explaining its functionality. Apart from these included modules, ActiveModel itself has a bunch of useful stuff to add to your Ruby classes that are really worth checking out.

This is the kind of thing that makes me a happier Rails developer every day. What about you, what makes you a happier Rails developer? Please take a moment to tell us in the comments section below :)

The Carnival is over in Brazil but we are still partying at Plataformatec by bringing you a complete new release of SimpleForm. This time is not a small bump though, it’s a shiny new version: SimpleForm 2.0, that comes with a bunch of new features and customizations, a new wrapper API to create custom input stacks and a great integration with Twitter Bootstrap.

Wrappers API

The new wrappers API is here in place of the old components option (besides some other *_tag and *_class configs), to add more flexibility to the way you build SimpleForm inputs. Here is an example of the default wrapper config that ships with SimpleForm when you run its install generator:

config.wrappers :default, :class => :input,
  :hint_class => :field_with_hint, :error_class => :field_with_errors do |b|
  ## Extensions enabled by default
  # Any of these extensions can be disabled for a
  # given input by passing: `f.input EXTENSION_NAME => false`.
  # You can make any of these extensions optional by
  # renaming `b.use` to `b.optional`.
  # Determines whether to use HTML5 (:email, :url, ...)
  # and required attributes
  b.use :html5
  # Calculates placeholders automatically from I18n
  # You can also pass a string as f.input :placeholder => "Placeholder"
  b.use :placeholder
  ## Optional extensions
  # They are disabled unless you pass `f.input EXTENSION_NAME => :lookup`
  # to the input. If so, they will retrieve the values from the model
  # if any exists. If you want to enable the lookup for any of those
  # extensions by default, you can change `b.optional` to `b.use`.
  # Calculates maxlength from length validations for string inputs
  b.optional :maxlength
  # Calculates pattern from format validations for string inputs
  b.optional :pattern
  # Calculates min and max from length validations for numeric inputs
  b.optional :min_max
  # Calculates readonly automatically from readonly attributes
  b.optional :readonly
  ## Inputs
  b.use :label_input
  b.use :hint,  :wrap_with => { :tag => :span, :class => :hint }
  b.use :error, :wrap_with => { :tag => :span, :class => :error }

Wrappers are used by the form builder to generate a complete input. You can remove any component from the wrapper, change the order or even add your own to the stack.

The :default wrapper is going to be used in all forms by default. You can also select which wrapper to use per form, by naming them:

# Given you added this wrapper in your SimpleForm initializer:
config.wrappers :small do |b|
  b.use :placeholder
  b.use :label_input
# Uses the :small wrapper for all inputs in this form.
simple_form_for @user, :wrapper => :small do |f|
  f.input :name

Or you can just pick a different wrapper in a specific input if you want:

# Uses the default wrapper for other inputs, and :small for :name.
simple_form_for @user do |f|
  f.input :name, :wrapper => :small

You can see a more detailed description of the new wrappers API in the documentation.

Twitter Bootstrap

The second big change in SimpleForm 2.0 is out of the box Bootstrap integration. SimpleForm now ships with a generator option to initialize your application with a set of specific wrappers customized for Bootstrap. To get them, just run in your terminal, inside a Rails application (with SimpleForm already installed):

rails generate simple_form:install --bootstrap

This gives you the default SimpleForm initializer in config/initializers/simple_form.rb with some extra integration code added for Bootstrap. For example, here is the default wrapper:

config.wrappers :bootstrap, :tag => 'div', :class => 'control-group', 
  :error_class => 'error' do |b|
  b.use :placeholder
  b.use :label
  b.wrapper :tag => 'div', :class => 'controls' do |ba|
    ba.use :input
    ba.use :error, :wrap_with => { :tag => 'span', :class => 'help-inline' }
    ba.use :hint,  :wrap_with => { :tag => 'p', :class => 'help-block' }

This wrapper is setup with the same structure that Bootstrap expects and is set to be the default wrapper in your application. This is the killer feature in SimpleForm 2.0: the Bootstrap integration is not inside SimpleForm but all in your application. This means that, if you want to move away or customize Bootstrap in the future, you don’t need to monkey patch SimpleForm, everything is in your app!

We’ve set up a live example application showing most of the SimpleForm inputs integrated with Twitter Bootstrap, make sure you check it out! The application code is on github.

Keep reading this blog post to find out the other changes and deprecations that gave SimpleForm all this extra flexibility, allowing it to be easily integrated with Twitter Bootstrap 2.0.

New configs

SimpleForm 2.0 comes with some new configs to ease its integration with Bootstrap and to make your daily work even more flexible:

  • default_wrapper: defines the default wrapper to be used when no one is given.
  • button_class: defines a class to add for all buttons.
  • boolean_style: change the way booleans (mainly check boxes and radio buttons) are shown: :inline (the default) uses the same structure as before, checkbox + label; :nested (generated for new apps) puts the checkbox inside the label, as label > checkbox.
  • collection_wrapper_class: class to add in all collections (check boxes / radio buttons), given collection_wrapper_tag is set.
  • item_wrapper_class: class to add to all items in a collection.
  • generate_additional_classes_for: allows you to specify whether to generate the extra css classes for inputs, labels and wrappers. By default SimpleForm always generate all classes, such as input type and required info, to all of them. You can be more selective and tell SimpleForm to just add such classes to the input or wrapper, by changing this config.


In order to create the new wrappers API, we had to deprecate some configs and change some helpers, so here is a basic summary of what is being deprecated:


  • translate: By making placeholder and hint optional options in the wrappers API, you can already disable the automatic translation attempt that happens for these components. labels, on the other hand, are always used in forms, so we added a special config for them: translate_labels.
  • html5: this config is now part of the wrappers API, with b.use :html5, so the config option has been deprecated.
  • error_notification_id: in favor of using error_notification_class only.
  • wrapper_tag=, wrapper_class=, wrapper_error_class=, error_tag=, error_class=, hint_tag=, hint_class=, components=: all these were moved to the wrappers API structure, and are not required anymore.


  • :radio input type: In order to integrate with Bootstrap, we had to get rid of the :as => :radio and use :as => :radio_buttons instead. The former still works, but will give you a bunch of deprecation warnings. CSS class names changed accordingly as well
  • collection_radio: has changed to collection_radio_buttons to follow the :as => :radio_buttons change. Its label class has changed as well based on the helper name.

Wrapping up

SimpleForm 2.0 comes with a lot of new features, in special the new wrappers API, to make it flexible enough to allow you to customize inputs as much as possible in an easier way, and to bring you the integrated Bootstrap structure.

Make sure you check out the new SimpleForm README and also the CHANGELOG for a full list of changes. We’ve also created an special wiki page to help you Upgrading to SimpleForm 2.0.

If you find any trouble while migrating to 2.0, or any issue with Bootstrap integration, or any other issue, please let us know in the issues tracker. And if you have any questions, make sure to send them to the mailing list, there are a lot of people there to help you.

All our development team and an amazing number of contributors put a lot of effort into this new release and we hope you will enjoy it. SimpleForm 2.0 + Bootstrap: from us, for you, with love.

Thoughts about SimpleForm 2.0? Please let us know in the comments.