Posts tagged "ruby"

Since moving to Mac, I always find it surprising that the default Mac OS X Ruby and IRB doesn’t allow inputs with accented characters. So, you cannot do

name = "George Guimarães"

This is annoying for brazilians and anyone who uses non-ASCII characters. The problem is that the default Ruby in Mac OS X isn’t linked against readline. A simple solution is to compile readline on your system, and then compile your own ruby binaries. A better one is to use tools that automate this process.

This post by Christopher Sexton uses homebrew to compile readline-0.6 but compiles ruby by hand. Since we love RVM, we’ll use it too.

If you use Homebrew (and you *should*), just do

brew install readline
brew link readline

Beware: linking readline into your system may break other tools that depends on readline source to compile. It was harmless on my system.

Ok, so now we want to build a new ruby and irb binaries. I wanted to use ruby 1.8.7-p248, so:

rvm install 1.8.7-p248 -C --enable-shared,--with-readline-dir=/usr/local

This tells the configure script to enable shared library linking (it is the default), and to search for readline in /usr/local (homebrew has just linked readline there). You may have to use --force if you already have this ruby version compiled. Newer versions of RVM do not need --force.

Now you can use accented and unicode characters on keyboard input in IRB using ruby 1.8.7.

And you? Do you have any tricks with IRB that you may want to share? Do you use Wirble, Utility Belt or others?

One of the beauties in the Open Source world is the possibility of reading other people source code and learn new things. However, lately I found out that not only the library code, but the test suite of several open source projects are full lessons for us.

In this post, I want to tell you which are the three test suites that I admire the most and why.

Integration award: Railties

Rails 3 has several improvements and not all of them may be visible to the application developer. One of the hidden unicorns is Railties test suite. As Yehuda stated in a blog post during the refactoring of version 2.3 to 3.0:

“Although the Rails initializer tests covered a fair amount of area, successfully getting the tests to pass did not guarantee that Rails booted.”

This happened because, in order to have fast tests, Rails 2.3 suite stubbed and mocked a significant part of the booting process. The new test suite is able to create a new application using the application generator, change configuration options, add plugins and engines, boot it and even make HTTP requests using Rack::Test.

For instance, take a look at this test which ensures that app/metals inside plugins are successfully added to the application middleware stack:

def test_plugin_metals_added_to_middleware_stack
  @plugin.write 'app/metal/foo_metal.rb', <<-RUBY
    class FooMetal
      def self.call(env)
        [200, { "Content-Type" => "text/html"}, ["FooMetal"]]
      end
    end
  RUBY
 
  boot_rails
  require 'rack/test'
  extend Rack::Test::Methods
 
  get "/not/slash"
  assert_equal 200, last_response.status
  assert_equal "FooMetal", last_response.body
end

The most important lesson here is: whenever mocking or stubbing in our tests, we still need to add tests without the mocks and stubs to ensure all API contracts are respected.

Readability award: Capybara

Capybara is a tool to aid writing acceptance tests for web applications. Capybara can use several drivers to interact with a web application, as Selenium, Celerity or even Rack::Test. Each driver needs a different setup and has different features. For instance, both Selenium and Celerity can handle javascript, but not Rack::Test.

As you may imagine, all these different drivers can make a test suite become a real spaghetti. However, Jonas Nicklas was able to transform a potential problem into a very elegant and readable test suite with Rspec help. Here is, for instance, the tests for selenium:

describe Capybara::Driver::Selenium do
  before do
    @driver = Capybara::Driver::Selenium.new(TestApp)
  end
 
  it_should_behave_like "driver"
  it_should_behave_like "driver with javascript support"
end

Each behavior group above (“driver” and “driver with javascript support”) is inside Capybara library allowing everyone to develop its own extensions using a shared suite. For instance, if a driver has javascript support, it means the following tests should pass:

shared_examples_for "driver with javascript support" do
  before { @driver.visit('/with_js') }
 
  describe '#find' do
    it "should find dynamically changed nodes" do
      @driver.find('//p').first.text.should == 'I changed it'
    end
  end
 
  describe '#drag_to' do
    it "should drag and drop an object" do
      draggable = @driver.find('//div[@id="drag"]').first
      droppable = @driver.find('//div[@id="drop"]').first
      draggable.drag_to(droppable)
      @driver.find('//div[contains(., "Dropped!")]').should_not be_nil
    end
  end
 
  describe "#evaluate_script" do
    it "should return the value of the executed script" do
      @driver.evaluate_script('1+1').should == 2
    end
  end
end

Capybara test suite is one of the best examples of using tests as documentation. By skimming the test suite you can easily know which features are supported by each driver! Sweet, isn’t it?

Friendliness award: I18n

When you are a big Open Source project, your test suite needs to be easy to run in order to new developers can create patches without hassle. The I18n library for Ruby definitely meets the big Open Source project requirement since it’s widely used and provides several extensions.

However, some of these extensions depends on ActiveRecord, some in ruby2ruby, others in ruby-cldr… and soon it will even support a few Key-Value stores, as Tokyo and Redis. Due to all these dependencies, you would probably imagine that running I18n test suite would require several trials and a lot of configuration before it finally works, right?

WRONG! If you don’t have ActiveRecord, I18n will say: “hey, you don’t have ActiveRecord” but still run the part of test suite that does not depend on it. So if a developer wants to fix or add something trivial, he doesn’t need to worry with installing all sorts of dependencies.

Besides, as mentioned a couple months ago, the I18n library allows you to create several combinations of backends. In other words, the I18n test suite needs to ensure that all these different combinations work as expected.

This problem is quite similar to the one in Capybara which needs to test different drivers. However, I18n uses Test::Unit thus it cannot use shared examples groups as in Rspec. So how were I18n developers able to solve this issue? Using Ruby modules!

Here are the tests for the upcoming KeyValue backend:

require 'test_helper'
require 'api'
 
class I18nKeyValueApiTest < Test::Unit::TestCase
  include Tests::Api::Basics
  include Tests::Api::Defaults
  include Tests::Api::Interpolation
  include Tests::Api::Link
  include Tests::Api::Lookup
  include Tests::Api::Pluralization
  # include Tests::Api::Procs
  include Tests::Api::Localization::Date
  include Tests::Api::Localization::DateTime
  include Tests::Api::Localization::Time
  # include Tests::Api::Localization::Procs
 
  STORE = Rufus::Tokyo::Cabinet.new('*')
 
  def setup
    I18n.backend = I18n::Backend::KeyValue.new(STORE)
    super
  end
 
  test "make sure we use the KeyValue backend" do
    assert_equal I18n::Backend::KeyValue, I18n.backend.class
  end
end

Each included module above adds a series of tests to the backend. Since key-value backends cannot store procs, we don’t include any test related to procs.

Wrapping up

These three are my favorite test suites and also part of my favorite open source projects!

We’ve adopted Capybara as the official testing tool at PlataformaTec for some time already and I18n is one of the subjects of my upcoming book about Rails 3. In one specific chapter, we will build a tool that stores I18n translations into TokyoCabinet, which allows us to create and update translations through a web interface, similarly to ActiveRecord. The only difference is that TokyoCabinet is waaaay faster.

Finally, the fact you can mimic several of Rspec features using simple Ruby (like Capybara using shared example groups and I18n simply using modules) will be part of my talk in Euruko 2010 entitled DSL or NoDSL: The power is in the middle. The talk will show cases where DSLs mimics much of the behavior provided by Ruby and discuss what we are winning and/or losing in such cases.

Keep following us and, until the next blog post is out, we would love to hear in the comments which are your favorite test suites!

This weekend during Rails Bugmash I stumbled across some nice posts about Rails 3 generators which motivated me to share them and add some comments!

First, David Trasbo wrote a nice guide about how to make your first Rails 3 generator, it covers up all the basic steps including setting it up in a gem. He also puts the deserved attention into Thor::Actions, which contains most helpers you need in a generator, like copy_file, template, create_file, empty_directory and so on.

On another post, Ben Scofield talks about apply method, which is also in Thor::Actions, and how to use it to dry your application templates.

Wait… so Thor::Actions is used both in generators and in Rails application templates? Exactly, Rails’ new generators unifies both application templates and generators API into one. While Thor::Actions holds basic methods, all Rails specific methods like environment, rakefile, generator are in Rails::Generators::Actions. If you already wrote an application template, you will feel more at home when writing a Rails 3 generator.

Paul Barry talks how easy it’s to customize your scaffold to use Rspec, Haml and Factory Girl instead of Test::Unit, Erb and Fixtures. This all works because scaffold is just a meta generator which provides hooks for template engine, test framework, ORM and so forth. A good way to see the hooks system working is by running script/generate scaffold --help before and after Paul changes, so you can see exactly how generators options update depending on the configuration values you set. While I wrote Rspec generators used in the example, he implemented himself Haml and Factory Girl generators and they can all be used as example if you plan to build your own.

Finally, Zigzag Chen wrote about templates customization and how you can change your scaffold controller to use json instead of the xml format. New generators have source paths, so you can customize them simply by copying files to RAILS_ROOT/lib/templates.

Rails Bugmash was excellent to gather feedback and we also got some tickets on Lighthouse, mainly about how generators help can be improved for people starting with Rails. Many thanks to Rails Bridge and besides the posts linked above, there is a generator guide, which may help you get started and maybe write your own post as well. :)

Pratik lately is doing a great work refactoring ActiveRecord to make a full use of relations. Speaking in code language, this means that in Rails 3 you will be able to do:

  User.where(:name => "Jose").order("birthday")

And it will return a relation. The relation can be further manipulated and the query is only triggered when you call all, first, to_a or some Enumerator method.

Besides that, he’s also doing some crazy experiments, which will probably become a plugin later. While discussing with Pratik some ways to implement equality and inequality, I discovered a neat ruby trick. Open up an irb session and do:

MD7M9TSHSAJY
~2  #=> -3
~42 #=> -43

And this method is actually documented.

The nice thing though, is that you can define this method in other classes as well. In the querying scenario for example, we could add this behavior:

class Symbol
  def ~
    :"LOWER(#{self})"
  end
end

And now we could actually do:

User.where(~:email => "jose@plataformatec.com.br")
#=> SELECT * WHERE LOWER(email) = "jose@plataformatec.com.br"

Unary operators

Yehuda Katz later pointed to me that this can actually be done with any of the three unary operators in Ruby: +, – and ~.

So we could also do:

class String
  def +@
    upcase
  end
 
  def -@
    downcase
  end
end

And now:

+"jose" #=> "JOSE"
-"JOSE" #=> "jose"

You could even “go crazy” and append several unary operators:

+-+"jose" #=> "JOSE"

And you? Did you know such behavior in Ruby? Do you have any good use case for it?

This summer I was selected with Josh Peek, Emilio Tagua and Nelson Crespo to work with Rails on Google Summer of Code (GSoC), which Nelson named as the Rails Summer Quartet. :)

Here, at Plataforma, we use a set of tools on our projects, which includes Inherited Resources, Remarkable and Formtastic. At some point, we were planning on creating generators for each of those tools but still they couldn’t play along. If I wanted a generator that uses all three projects, I needed to create a inherited_remarkable_formtastic generator which is not DRY at all.

For example, for those who wants to use Datamapper with Rspec, they need to call “ruby script/generate dm_rspec_model” instead of “ruby script/generate model”. Since Rails 3.0 is moving towards agnosticism, my GSoC proposal was exactly bring it to rails generators.

1. So, what about Thor?!

One day before the official coding period start, I was staring at this Thor post by Yehuda Katz. Thor is a rake replacement with support to options parsing:

class Speak < Thor
  desc "name", "the name to say hello to"
  method_options :loudly => false
 
  def hello(name)
    name.upcase! if options[:loudly]
    puts "Hello #{name}"
  end
end

And then can be invoked as:

> thor speak:hello jose
Hello jose
 
> thor speak:hello jose –-loudly
Hello JOSE

At that point, I realized that a generator is nothing more than a scripting task (like rake or thor) with some extra methods which makes the creation and copy of files easy. Thor had several features which convinced me that it was the best solution to build generators on top of:

  • It has a powerful options parser;
  • Thor classes can be inherited and all tasks from the class are copied to the child;
  • Thor classes are self contained. The example above can be invoked straight from your ruby code as Speak.new.hello(“jose”).

Then I was able to create a ROADMAP to Thor:

  • Add actions like copy_file, template, empty_directory to Thor;
  • Move all user input and output logic to its own class, so anyone can customize it;
  • Extend even more Thor options parser to add support to hashes (as in Rails name:string age:integer on generators) and arrays;
  • Add an invocation mechanism, so I can invoke one task from another Thor task.

2. An example

Let’s see an example on how you can create your own generators with Thor. For example, a generators that stubs out a new gem structure:

class NewgemGenerator < Thor::Group
  include Thor::Actions
 
  # Define arguments and options
  argument :name
  class_option :test_framework, :default => :test_unit
 
  def self.source_root
    File.dirname(__FILE__)
  end
 
  def create_lib_file
    create_file "#{name}/lib/#{name}.rb" do
      "class #{name.camelize}\nend"
    end
  end
 
  def create_test_file
    test = options[:test_framework] == "rspec" ? :spec : :test
    create_file "#{name}/#{test}/#{name}_#{test}.rb"
  end
 
  def copy_licence
    if yes? "Use MIT license?"
      # Make a copy of the MITLICENSE file at the source root
      copy_file "MITLICENSE", "#{name}/MITLICENSE"
    else
      say "Shame on you…", :red
    end
  end
end

You can see from the example above that we are inheriting from Thor::Group and not Thor. In Thor, each method corresponds to a task, which can be invoked on its own. In Thor::Group, you invoke all tasks at once, in the order they are declared. This is interesting because you split your script/generator into several methods. It improves readability and allows anyone to inherit from your generator and change just one step in the process.

The next step, on lines 4 and 5, is to define arguments and options for the class. Arguments are required to be given right after the executable while options are given with switches. The newgem above can be invoked as:

newgem remarkable

And it will create two files: “remarkable/lib/remarkable.rb”, “remarkable/test/remarkable_test.rb” and prompt the user (with the use of the method yes?) if we wants to copy the MITLICENSE. If you want to change the test framework, you can give it as an option:

newgem remarkable --test-framework=rspec

Now it generates “remarkable/lib/remarkable.rb” and “remarkable/spec/remarkable_spec.rb”.

The generation methods are kept into the Thor::Actions module, which is included on top of our class. It holds all the scripting methods, which are: copy_file, create_file, directory, empty_directory, get, inject_into_file and template. All those actions can be revoked, so Thor knows how to do and undo the process (like in script/generate and script/destroy).

Even more, some of Rails templates methods was moved to Thor, like: inside, run, run_ruby_script, gsub_file, append_file and prepend_file. So whenever creating scripts with Thor, those methods will be available to make your life easier.

Finally, all user iteration methods are handled by Thor::Shell classes by say, ask, yes? and no? methods. Thor ships with two Shell implementations: Basic and Color. If you mantain an IDE for Rails, you can build your own shell and make the user interaction through it.

3. What is more?

Thor is used as base in Rails::Generators, where Rails extend it to provide Rails specific functionalities, as hooks for ORM, Test framework and so on. This will be my talk subject on Rails Summit Latin America, 13th October, in São Paulo, Brazil.

If you can join us or not, be sure to grab our RSS feed and keep on checking, we will discuss about it here too.