PostgreSQL Arrays and Rails Migrations

Suppose we have a Product model with the following fields: name, category_id and tags. The name field will be a simple string, category_id will be the foreign key of a record in the Category model and tags will be created by inputting a string of comma-separated words, so: “one, two, forty two” will become the tags: “one”, “two” and “forty two” respectively.

Creating these tables via migrations is nothing new, except for the column tags which will have the Array type in this case. To create this kind of column we use the following syntax in our migration:

create_table :categories do |t|
  t.string :name, null: false
end

create_table :products do |t|
  t.string :name, null: false
  t.references :category, null: false
  t.text :tags, array: true, default: []
end

Let’s explore what we can do with this kind of field using the postgres console:

$ rails db
> insert into products(name, category_id, tags) values('T-Shirt', 3, '{clothing, summer}');
> insert into products(name, category_id, tags) values('Sweater', 3, ARRAY['clothing', 'winter']);
> select * from products;
1  |  T-Shirt  |  3  | {clothing, summer}
2  |  Sweater  |  3  | {clothing, winter}

As we can see we need to specify each tag following this syntax:

‘{ val1, val2, … }’ or ARRAY[‘val1’, ‘val2’, …]

Let’s play a little more to understand how this column behaves when queried:

> select * from products where tags = '{clothing, summer}';
1  |  T-Shirt  |  3  | {clothing, summer}

> select * from products where tags = '{summer, clothing}';
(0 rows)

> select * from products where 'winter' = ANY(tags);
2  |  Sweater  |  3  |  {clothing, winter}

As this example demonstrates, searching for records by an array with its values in the order they were inserted works, but with the same values in a different order does not. We were also able to find a record searching for a specific tag using the ANY function.

There’s a lot more to talk about arrays in PostgreSQL, but for our example this is enough. You can find more information at the PostgreSQL official documentation about arrays and its functions.

How Rails treats PostgreSQL arrays

It’s also valuable to see how to use the array field within Rails, let’s try:

$ rails c

Product.create(name: 'Shoes', category: Category.first, tags: ['a', 'b', 'c'])
#> 

Product.find(26).tags
#> ["a", "b", "c"]

So Rails treats an array column in PostgreSQL as an Array in Ruby, pretty reasonable!

Validations

We want each product to be unique, let’s see some examples to clarify this concept.

Given we have the following product:

Product.create(name: 'Shoes', category: Category.first, tags: ['a', 'b', 'c'])

We can easily create another one if we change the name attribute:

Product.create(name: 'Slippers', category: Category.first, tags: ['a', 'b', 'c'])

We can also create another product with different tags:

Product.create(name: 'Shoes', category: Category.first, tags: ['a', 'b'])

But we don’t want to create a product with the same attributes, even if the tags are in a different order:

Product.create(name: 'Shoes', category: Category.first, tags: ['a', 'c', 'b'])
#> false

As PostgreSQL only finds records by tags given the exact order in which they were inserted, then how can we ensure the uniqueness of a product with tags in an order-independent way?

After much thought we decided that a good approach would involve creating an unique index with all the columns in the products table but with tags sorted when a row is inserted in the database. Something like:

CREATE UNIQUE INDEX index_products_on_category_id_and_name_and_tags
ON products USING btree (category_id, name, sort_array(tags));

And sort_array is our custom function responsible for sorting the array, since PostgreSQL does not have a built in function like this.

Creating a custom function in PostgreSQL using PL/pgSQL

To create a custom function we used the PL/pgSQL language, and since we are adding database specific code like this we can’t use the default schema.rb anymore. Let’s change this in config/application.rb:

# Use SQL instead of AR schema dumper when creating the database
config.active_record.schema_format = :sql

With this configuration set, our schema.rb file will be replaced by a structure.sql file without side effects, our current migrations don’t need to be changed at all. Now we can create a migration with our sort_array code:

def up
  execute <<-SQL
    CREATE FUNCTION sort_array(unsorted_array anyarray) RETURNS anyarray AS $$
      BEGIN
        RETURN (SELECT ARRAY_AGG(val) AS sorted_array
        FROM (SELECT UNNEST(unsorted_array) AS val ORDER BY val) AS sorted_vals);
      END;
    $$ LANGUAGE plpgsql IMMUTABLE STRICT;

    CREATE UNIQUE INDEX index_products_on_category_id_and_name_and_tags ON products USING btree (category_id, name, sort_array(tags));
  SQL
end

def down
  execute <<-SQL
    DROP INDEX IF EXISTS index_products_on_category_id_and_name_and_tags;
    DROP FUNCTION IF EXISTS sort_array(unsorted_array anyarray);
  SQL
end

Now, let's take it slow and understand step by step

CREATE FUNCTION sort_array(unsorted_array anyarray) RETURNS anyarray

The line above tells that we are creating a function named sort_array and that it receives a parameter named unsorted_array of type anyarray and returns something of this same type. This anyarray, in fact, is a pseudo-type that indicates that a function accepts any array data type.

RETURN (SELECT ARRAY_AGG(val) AS sorted_array
FROM (SELECT UNNEST(unsorted_array) AS val ORDER BY val) AS sorted_vals);

The trick here is the use of the function unnest that expands an Array to a set of rows. Now we can order these rows and after that we use another function called array_agg that concatenates the input into a new Array.

$$ LANGUAGE plpgsql IMMUTABLE STRICT;

The last trick is the use of the keywords IMMUTABLE  and STRICT. With the first one we guarantee that our function will always return the same output given the same input, we can't use it in our index if we don't specify so. The other one tells that our function will always return null if some of the parameters are not specified.

And that's it! With this we can check for uniqueness in a performant way with some method like:

def duplicate_product_exists?
  relation = self.class.
    where(category_id: category_id).
    where('lower(name) = lower(?)', name).
    where('sort_array(tags) = sort_array(ARRAY[?])', tags)

  relation = relation.where.not(id: id) if persisted?

  relation.exists?
end

Case insensitive arrays

There is still a problem with our code though, the index is not case insensitive!  What if a user inserts a product with tags ['a', 'b'] and another one inserts the same product but with tags ['A', 'b']? Now we have duplication in our database! We have to deal with this, but unfortunately this will increase the complexity of our sort_array function a little bit. To fix this problem we only need to change one single line:

From this:

FROM (SELECT UNNEST(unsorted_array) AS val ORDER BY val) AS sorted_vals);

To:

FROM
(SELECT
  UNNEST(string_to_array(lower(array_to_string(unsorted_array, ',')), ','))
  AS val ORDER BY val)
AS sorted_vals);

The difference is that instead of passing unsorted_array directly to the function unnest we are transforming it in an String, calling lower on it and transforming it back to an Array before passing it on. With this change it doesn't matter if the user inserts ['a'] or ['A'], every tag will be saved in lowercase in the index. Problem solved!

As we can see, it's not an easy task to deal with uniqueness and arrays in the database, but the overall result was great.

Would you solve this problem in a different way? Share with us!

The post Rails 4 and PostgreSQL Arrays first appeared on Plataformatec Blog.

This rc version took some time to get ready, we’ve been running a rails4 branch for some time already and one of the reasons was because of the changes required to make it compatible with the new strong parameters API from Rails 4. We are aware that some people have been using this branch since Rails 4.0 beta1 with success, and we’re now inviting you to try 3.0 rc with the recent release of Rails 4.0 rc1.

Devise stable

Together with the 3.0 beta version, we’ve released Devise 2.2.4 with a few enhancements and bug fixes, make sure to check the changelog to see the new goodies. All changes are also included in the rc version.

Simple Form

Simple Form has been running a 3.0 rc version for a couple months already, fully compatible with Rails 4 as well, and today we are releasing its release candidate version. In Simple Form master we just dropped support to the 3.x Rails series, focusing our work on Rails 4 compatibility from now on, due to a series of improvements in Rails 4 regarding form helpers – but don’t worry, we will be keeping a v2.1 branch with Rails 3.2 compatibility for a while.

We have some cool plans to improve the wrappers API even further, but that’s subject for another blog post :).

Responders

Responders has been around for quite some time already and we use it in most of our projects, so today we’re celebrating its 1.0 release candidate version, specially to support Rails 4.

Show For

Show For just got a new stable release, v0.2.6, with all the enhancements and bug fixes that were in master, plus a v0.3.0 rc version that adds Rails 4 support.

Mail Form

Mail Form also got a new 1.5 rc release with Rails 4.0 compatibility. Nothing else has changed from the current 1.4 version.

Has Scope

Has Scope is getting a new 0.6 rc version with Rails 4.0 compatibility, including a couple of fixes that were already present in master.

Compatibility

All these new releases are officially dropping support to Rails 3.0 and 3.1, and Ruby 1.8.7. We’ll keep compatibility with Rails 3.2 and 4.0 from now on, all of them on the same branches except for Simple Form which has different branches for each Rails version.

Wrapping up

We’ve got new hot releases for you to try out with Rails 4, please give them a try and let us know if you find any issue or have any feedback.

We’d also like to specially thank everyone involved in helping us getting these projects up and running in Rails 4, without you folks that’d have never been possible.

Enjoy <3

The post Devise and Rails 4 first appeared on Plataformatec Blog.

Defining a scope

First of all, lets get a better understanding about how scopes are used. In Rails 3 you can define a scope in two ways:

class Post < ActiveRecord::Base
  scope :published, where(status: 'published')
  scope :draft, -> { where(status: 'draft') } 
end

The main difference between both usages is that the :published condition is evaluated when the class is first loaded, whereas the :draft one is lazy evaluated when it is called. Because of that, in Rails 4 the first way is going to be deprecated which means you will always need to declare scopes with a callable object as argument. This is to avoid issues when trying to declare a scope with some sort of Time argument:

class Post < ActiveRecord::Base
  scope :published_last_week, where('published_at >= ?', 1.week.ago)
end

Because this won’t work as expected: 1.week.ago will be evaluated when the class is loaded, not every time the scope is called.

Scopes are just class methods

Internally Active Record converts a scope into a class method. Conceptually, its simplified implementation in Rails master looks something like this:

def self.scope(name, body)
  singleton_class.send(:define_method, name, &body)
end

Which ends up as a class method with the given name and body, like this:

def self.published
  where(status: 'published')
end

And I think that’s why most people think: “Why should I use a scope if it is just syntax sugar for a class method?”. So here are some interesting examples for you to think about.

Scopes are always chainable

Lets use the following scenario: users will be able to filter posts by statuses, ordering by most recent updated ones. Simple enough, lets write scopes for that:

class Post < ActiveRecord::Base
  scope :by_status, -> status { where(status: status) }
  scope :recent, -> { order("posts.updated_at DESC") }
end

And we can call them freely like this:

Post.by_status('published').recent
# SELECT "posts".* FROM "posts" WHERE "posts"."status" = 'published' 
#   ORDER BY posts.updated_at DESC

Or with a user provided param:

Post.by_status(params[:status]).recent
# SELECT "posts".* FROM "posts" WHERE "posts"."status" = 'published' 
#   ORDER BY posts.updated_at DESC

So far, so good. Now lets move them to class methods, just for the sake of comparing:

class Post < ActiveRecord::Base
  def self.by_status(status)
    where(status: status)
  end
  
  def self.recent
    order("posts.updated_at DESC")
  end
end

Besides using a few extra lines, no big improvements. But now what happens if the :status parameter is nil or blank?

Post.by_status(nil).recent
# SELECT "posts".* FROM "posts" WHERE "posts"."status" IS NULL 
#   ORDER BY posts.updated_at DESC

Post.by_status('').recent
# SELECT "posts".* FROM "posts" WHERE "posts"."status" = '' 
#   ORDER BY posts.updated_at DESC

Oooops, I don't think we wanted to allow these queries, did we? With scopes, we can easily fix that by adding a presence condition to our scope:

scope :by_status, -> status { where(status: status) if status.present? }

There we go:

Post.by_status(nil).recent
# SELECT "posts".* FROM "posts" ORDER BY posts.updated_at DESC

Post.by_status('').recent
# SELECT "posts".* FROM "posts" ORDER BY posts.updated_at DESC

Awesome. Now lets try to do the same with our beloved class method:

class Post < ActiveRecord::Base
  def self.by_status(status)
    where(status: status) if status.present?
  end
end

Running this:

Post.by_status('').recent
NoMethodError: undefined method `recent' for nil:NilClass

And :bomb:. The difference is that a scope will always return a relation, whereas our simple class method implementation will not. The class method should look like this instead:

def self.by_status(status)
  if status.present?
    where(status: status)
  else
    all
  end
end

Notice that I'm returning all for the nil/blank case, which in Rails 4 returns a relation (it previously returned the Array of items from the database). In Rails 3.2.x, you should use scoped there instead. And there we go:

Post.by_status('').recent
# SELECT "posts".* FROM "posts" ORDER BY posts.updated_at DESC

So the advice here is: never return nil from a class method that should work like a scope, otherwise you're breaking the chainability condition implied by scopes, that always return a relation.

Scopes are extensible

Lets get pagination as our next example and I'm going to use the kaminari gem as basis. The most important thing you need to do when paginating a collection is to tell which page you want to fetch:

Post.page(2)

After doing that you might want to say how many records per page you want:

Post.page(2).per(15)

And you may to know the total number of pages, or whether you are in the first or last page:

posts = Post.page(2)
posts.total_pages # => 2
posts.first_page? # => false
posts.last_page?  # => true

This all makes sense when we call things in this order, but it doesn't make any sense to call these methods in a collection that is not paginated, does it? When you write scopes, you can add specific extensions that will only be available in your object if that scope is called. In case of kaminari, it only adds the page scope to your Active Record models, and relies on the scope extensions feature to add all other functionality when page is called. Conceptually, the code would look like this:

scope :page, -> num { # some limit + offset logic here for pagination } do
  def per(num)
    # more logic here
  end
  
  def total_pages
    # some more here
  end
  
  def first_page?
    # and a bit more
  end
  
  def last_page?
    # and so on
  end
end

Scope extensions is a powerful and flexible technique to have in our toolchain. But of course, we can always go wild and get all that with class methods too:

def self.page(num)
  scope = # some limit + offset logic here for pagination
  scope.extend PaginationExtensions
  scope
end

module PaginationExtensions
  def per(num)
    # more logic here
  end
  
  def total_pages
    # some more here
  end
  
  def first_page?
    # and a bit more
  end
  
  def last_page?
    # and so on
  end
end

It is a bit more verbose than using a scope, but it yields the same results. And the advice here is: pick what works better for you but make sure you know what the framework provides before reinventing the wheel.

Wrapping up

I personally tend to use scopes when the logic is very small, for simple where/order clauses, and class methods when it involves a bit more complexity, but whether it receives an argument or not doesn't really matter much to me. I also tend to rely more on scopes when doing extensions like showed here, since it's a feature that Active Record already gives us for free.

I think it's important to clarify the main differences between scopes and class methods, so that you can pick the right tool for the job, or the tool that makes you more comfortable. Whether you use one or another, I don't think it really matters, as long as you write them clear and consistently throughout your application.

Do you have any thought about using scopes vs class methods? Make sure to leave a comment below telling us what you think, we'd love to hear.

The post Active Record scopes vs class methods first appeared on Plataformatec Blog.

Abaixo você pode ver os temas, com links para os slides e vídeos:

Palestras

Vamos falar sobre Concorrência

Por José Valim. Confira o vídeo.

Escrevendo Aplicações Melhores com Active Model

Por Carlos Antonio. Confira o vídeo.

Conhecendo as Entranhas do Rails

Por Rafael França. Confira o vídeo.

Lightning Talks

Contribuindo para o Rails

Por Carlos Galdino.

I18nAlchemy

Por Lucas Mazza.

Copyright, Licenças Open Source e você!

Por George Guimarães.

Confira o vídeo das Lightning Talks.

Sinta-se à vontade para ver e rever os slides e vídeos das palestras, e nos passar seu feedback através dos comentários. Não deixe também de conferir as outras palestras disponíveis, e se você escreveu um post sobre o evento em seu blog, adoraríamos ver um comentário com um link compartilhando seu post.

Gostaríamos também de agradecer e parabenizar o Fábio Akita e a Locaweb pela ótima organização e alta qualidade do evento, tudo funcionou perfeitamente para que todos pudessem aproveitar ao máximo a conferência.

E nos vemos na RubyConf Brasil 2013!

The post Talks da RubyConf Brasil 2012 first appeared on Plataformatec Blog.

No dia 31, às 9h40, José Valim falará na Sala 1 sobre concorrência e sobre o papel e a importância disso no desenvolvimento de aplicações.

Às 11h, na sala 2, Carlos Antonio falará sobre como usar Active Model para escrevermos aplicações melhores.

Por fim, às 11h50, na sala 1, Rafael França falará sobre as entranhas do Rails e as grandes novidades do Rails 4.

Não deixe também de votar em nossos Lightning Talks, que serão realizados no primeiro dia, a partir das 18h20:

• Contribuindo para o Rails, do Carlos Galdino
• I18nAlchemy, do Lucas Mazza
• Copyright, Licenças Open Source e você!, do George Guimarães

Nos vemos lá!

The post Plataformatec na RubyConf 2012 first appeared on Plataformatec Blog.

The content_for that we are used to

The content_for method is an old friend of every Rails developer, and it’s a pretty simple and flexible helper. You can store a chunk of HTML from a String or a block, and grab it somewhere else in your views or yield it directly into your templates. It’s a pretty handy trick to move data from your views into your layouts, like page titles, custom meta tags or specific script tags that your page needs to include.

# On your 'application.html.erb' layout, inside the '' tag.
<%= yield :metatags %>

# Then, into a specific view
<% content_for :metatags do %>
  
<% end %>

Multiple calls of the content_for helper using the same identifier will concatenate them and output them together when you read it back on your views, as:

<% content_for :example, "This will be rendered" %>
<% content_for :example do %>
  
This will be rendered too!
<% end %>

On some scenarios this behavior might not be desired, and with Rails 4 you can flush out the stored pieces of an identifier and replace it instead of adding more content to it: using the flush: true option. The first implementation used an extra true argument, but we changed to use a Hash instead, so the flush key can express better the behavior we’re expecting.

<% content_for :example, "This will be rendered" %>
<% content_for :example, flush: true do %>
  
But this will override everything on the ':example' block.
<% end %>

The gallery situation

I’ve stumbled upon this on a recent project, where we had a somewhat classic scenario: a partial named _gallery, responsible for rendering the piece of HTML to display a gallery of images that also supplies a content_for block with a script tag to include the required libraries to put the gallery to work.

  

<% content_for :scripts, javascript_include_tag('gallery') %>

It works like a charm. But with an updated requirement we had the case where multiple galleries could be present on the same page, rendering the _gallery partial several times. The required HTML would be present, but the gallery.js script would be included multiple times into the rendered page. Instead of working this out using instance variables to check that the partial was rendered at least once, we could let Rails do all the hard work for us, using the flush option when including the gallery.js script.

  

<% # We can render this partial several times and this script will be included just once %>
<% content_for :scripts, javascript_include_tag('gallery'), flush: true %>

Back to the present: Rails 3.2

Well, while this seems to be a perfect solution to my problem, this feature isn’t available on Rails 3.2 or on the 3-2-stable branch – it’s only available on the master branch that will be released with Rails 4. But, backporting this feature into a 3.x application is pretty simple, using a helper of your own.

def single_content_for(name, content = nil, &block)
  @view_flow.set(name, ActiveSupport::SafeBuffer.new)
  content_for(name, content, &block)
end

After some source diving into the ActionPack source code we’re done – it just needs to replace any present content with a brand new SafeBuffer instance before storing the piece of HTML.

What do you think about this little addition to Rails 4? Can you think of a similar problem that could be solved with this instead of a custom hack?

The post Flushing content blocks with Rails 4 first appeared on Plataformatec Blog.