Posts tagged "ruby"

Rails 4.1 was just released this week and I already had a great experience trying out the release candidates on my latest project, so I decided to write a bit about my favorites features on this release and some things I have learned by using them so far.

1) secrets.yml

Placing your configuration in a YAML file isn’t exactly a revolutionary feature, but the usage of the config/secrets.yml file that comes with Rails 4.1 holds a more important idea: the promise of a default approach for environment aware custom configuration on Rails applications. Over the years the community created several ways to manage such configuration so every app out there deals with this differently, but now we can use the Rails default as a standard just like we do with the app folder or the routing patterns, taking the configuration madness outside the list of things to worry about when working with Rails. So instead of dealing with multiple YAML files or constants left out inside initializers, we can go with the secrets.yml as the default for our apps.

Remember that you can place any kind of configuration – not just secrets like tokens or passwords – that need to be handled differently through your application environments, like API Endpoints or S3 bucket names. And for any gem maintainers out there, you can make your gem read these settings from the secrets.yml automagically through an initializer block and maybe remove a configuration step from the gem setup. Adding this to Devise on this pull request was easier than I expected an I suggest you to try it out on your gems as well.

If you want to try to organize your configuration through the secrets.yml without having to update to Rails 4.1 right now, Andrew White backported this feature on the rails-secrets gem for Rails 4.0 apps.

So, if you are dealing with some configuration mess or aren’t using something like dotenv for your application, I strongly suggest that you try to migrate your config to use the secrets.yml file and see how it goes for your application.

2) Action Pack Variants

Variants are proving to be a great solution to render device specific views when mixed with any device detection solution like the useragent or browser gems, which you integrate quickly with just a before_action block:

class ApplicationController < ActionController::Base
  before_action :set_variant
 
  private
 
  def set_variant
    if browser.tablet?
      request.variant = :tablet
    elsif browser.mobile?
      request.variant = :mobile
    else
      request.variant = :desktop
    end
  end
end

Even though the main examples are dealing with User Agent sniffing, this feature can be used in any context where you want to have more control of which views are rendered by your application, like:

  • A/B Testing different partials based on the user cookies.
  • API versioning for your Jbuilder templates.
  • Maintaining current and redesigned views for the same controller.
  • Authorization aware views, like users/index.html+admin.erb or products/show.html+guest.erb.

In the end, Variants are just a way for you to have more control over how your views will be used by the app, helping you to remove boilerplate logic from your code and letting the framework handle it through a more elegant solution.

3) The improved cookies serializer

The changes on how Rails serializes cookies are a great improvement when it comes to security and stability of web apps. Before this, any object placed in the cookies Hash would be serialized (and deserialized) through Marshal.dump and Marshal.load, which could possibly lead to remote code execution if an attacker got hold on your application secret.

Now this serializer is configurable through the config.action_dispatch.cookies_serializer configuration option, and new apps will ship with a smarter default: a JSON serializer that won’t recreate complex objects besides Strings, Integers and other JSON data types. And for a smooth upgrade, you can use the :hybrid serializer that will convert your existing marshalled cookies into JSON cookies, so this upgrade can be transparent for your application users.

This upgrade highlights a possible bad practice in our applications where we end up placing more complex objects in the session that can’t be completely restored by the JSON serializer, when we should be using more simple structures for the data stored in cookies. Thanks to a related issue reported on the Devise issue tracker we could simplify the gem code a bit, so instead of serializing Time objects we could work with numbers.

So, when updating your application to use the :hybrid serializer, don’t forget to do a double check of whatever kind of data the app stores in your users cookies and look for possible backwards incompatibility. And if you want to take a closer look on how this was implemented, be sure to check the related issues and pull requests on the Rails repo: #12881, #13692 and #13945.

Keeping up to date with the latest Rails changes

Following the activity on the Rails repository over GitHub helped a lot to understand better these features and the rationale behind their implementations, but going through all the commits and discussions on Issues and Pull Requests would demand a lot of your time. If you want some of the inside scoop but don’t have that much time to go through the Rails activity over the week, Godfrey Chan has put up a weekly digest about Rails named This Week in Rails. I suggest that you subscribe to the list and even check some of the previous editions on the archive page.

Try it yourself!

Take some time and upgrade one of your Rails 4 apps and try out some of the new features! I bet that some of them will help you improve your codebase or make your coworkers life a bit easier, and we are eager to hear from your experience with the 4.1 release.

Setting up a machine for development is always a manual, annoying task. Here at Plataformatec we have a lot of projects that have different software dependencies, so every time one of our members has to start working on a new project, he needs to setup his machine to meet the project’s requirements, just like every team member of that project already did.

As such, we always end up with some questions: Is this project using MySQL, PostgreSQL or MongoDB? Do I need to install a JavaScript runtime? What about QT? Which Ruby version do I need?

Every team member joining an existing project gets stuck on those same questions, while all they want is to do what they do best, write software.

The problem is even worse when the person is a newcomer, as he may not have any previous knowledge about which software we use.

So, to help our team mates, the newcomers and to make easier and faster to get things done, we started to automate this setup, initially working in our own tool until the day GitHub released Boxen.

About Boxen

Boxen is a tool for automating your machine setup. It works on top of the Puppet project and has some utilities and default modules to make easier to provide and set up services and projects.

Also, Boxen gives you a way to define a personal manifest where you can put personal configurations like you favorite text editor, setting up your dotfiles, or other utility softwares.

How we use Boxen

As a consultancy company we have more than one project running at the same time. Besides that, one of our practices is team rotation, which means that every now and then people move to different teams. Boxen helped us to make those exchanges smoother.

Given that one person has already set up his project to use Boxen, the next person who joins the team will only have to run one command to get his machine ready to work.

The same occurs with problems that someone may have when setting up his machine. Once someone fixes the issue, everyone on the team will take advantage of that fix.

Right now we have some projects set up to use Boxen, and some of them are open source projects like Rails, Simple Form and Elixir.

Our Rails project configuration is open source for those who are interested to contribute to Rails.

Boxen is not only useful for tech people. Every person in the company can use it. Right now we have 10 people using Boxen, where 9 of them are developers and 1 is a project manager.

Conclusion

We have found Boxen to be very useful for our company and we have made it our default tool to set up our machines. Also, we think it is a great way to spread and perpetuate knowledge across your team about the project’s infrastructure and configuration.

I, in particular, learned a lot about systems operation and configuration in the path of making Boxen part of our toolset. Also, I’m very happy to say that I have been granted commit access to the Boxen organization and we will help the GitHub guys and the Boxen team to move this project forward.

And you, what is your team using to set up your machines?

Today I want to show you a project I’ve started over a year ago, during Mendicant University core skills course. For those who don’t know, Mendicant University is a group of skilled software developers that offer courses, mentoring, and help out the community, started by Gregory Brown, and that nowadays counts with some other awesome folks as part of the staff. I highly recommend taking a look at and enrolling.

Back to I18n, during Mendicant University we were supposed to create a project in Ruby, not specifically with Rails, and I decided to scratch my own itch by trying to solve a problem we usually have in Brazil: receiving date/time/numeric input from user interface. I know and have already used the delocalized gem, and it works quite nice, but sometimes I felt a bit uncomfortable about how it handled some parts of localization/parsing. This is mainly due to the need to monkey patch both ActiveRecord to handle input, and ActionView to handle output. Besides that, and most important, I had to come up with some project and I thought that’d be a good challenge :D.

The main goal of this project is to provide a proxy object to use with your ORM (currently ActiveRecord only) that will be responsible for localizing and parsing the date/time/numeric attributes when getting or setting their values, respectively. Lets see some quick examples:

# Include the proxy in your model
class Product < ActiveRecord::Base
  include I18n::Alchemy
end
 
# Grab your object from the database
@product   = Product.first
# Instantiate the localized proxy
@localized = @product.localized

Now that we have a localized proxy for the @product object, we can get/set numeric attributes with localized values, such as:

@localized.price = "1.99"
 
@product.price   # => 1.99
@localized.price # => "1.99"
 
I18n.with_locale :pt do
  @localized.price = "1,88"
 
  @product.price   # => 1.88
  @localized.price # => "1,88"
end

And also date/time attributes, for instance:

@localized.released_at = "12/31/2011"
 
@product.released_at   # => Date.new(2011, 12, 31)
@localized.released_at # => "12/31/2011"
 
I18n.with_locale :pt do
  @localized.released_at = "31/12/2011"
 
  @product.released_at   # => Date.new(2011, 12, 31)
  @localized.released_at # => "31/12/2011"
end

I18n Alchemy can also receive a hash of attributes, the same way you use with your models when calling new. That means you can use it like this:

# You could be using params[:product] for instance.
I18n.with_locale :pt do
  @localized = @product.localized(:price => "1,88") 
 
  @product.price   # => 1.88
  @localized.price # => "1,88"
end

The parsing/localization formats are basically the same ones you already use in your Rails application. You can check the basic locale configuration for I18n Alchemy in its README on github.

Wrapping up

I18n Alchemy is a small and new project which solves most of the problems we commonly face when dealing with localization and parsing of date/time/numeric values. It is tested with Rails 3.0, 3.1 and 3.2 and works with all the basic methods, such as attributes=, assign_attributes, update_attributes and nested attributes as well.

It was a really fun time creating it during Mendicant University, and it took a long time until I decided to release it as a gem. There is still a bunch of things to do, but I wanted to ask you to give it a try and let me know about any feedback you have.

As a side note, if you are interested in knowing more about the design decisions that led this project, you may want to take a look at Gregory Brown’s post on Ruby Best Practices, entitled “Issue #23: SOLID Design Principles”, more specifically in the Open/closed principle topic.

I’m releasing the first 0.0.1 version today, and I hope you find it useful. Have any comments? Let us know!

Outpost is me, scratching my own itch. For a while now, freelancing or working
for others, I work with pretty much all the stack in web development: from front-end
development with HTML and CSS up to the system administration.

And sometimes, I screw up. Sometimes, stuff go wrong. Maybe that Sphinx
isn’t getting indexed or even running after an unsuccessful deploy. Maybe that
background job isn’t being run, and I forget to verify it.

I know there are excellent tools that already solve this problem. But I wanted an
easy way I would be able to code my own monitoring rules, using one of the
languages I like the most. And also, the fantastic Aaron Patterson (or
tenderlove), one of the most prominent faces of the Ruby community once told in
a Q&A at RMU: it should be fun!

And thus the Outpost idea was born. Outpost is a framework so I can easily
implement Ruby code that query the current state of any service I want. I can
also write code that can go into the database and do a SELECT on a table (a
jobs table, for example) to check if everything’s fine.

What is Outpost?

Outpost is basically a DSL where you can describe rules to monitor your
service, application, server, whatever. Below is a very simple example:

require 'outpost'
require 'outpost/scouts'
 
class MyWebPageOutpost < Outpost::DSL
  using Outpost::Scouts::Http => "web page" do
    options :host => 'localhost', :port => 3000
    report :up, :response_code => 200
    report :down, :response_body => {:match => /Ops/}
  end
end

In this example, we are monitoring (using what I’ve called ‘Scouts’) HTTP
communication to localhost:3000. It will report that the system is
up if the response code is 200 (HTTP OK) and report that it is down if the
response body contains the word “Ops”, by matching a Regular Expression to it.

There is still a lot of work to be done, but I feel it is ready for a very first
release. There are only two Scouts today: HTTP and Ping, but it’s so
easy to write new ones that I will be releasing a few more in the next
days.

Also, there are only three expectation matchers: response time, response body
and response code. I believe they are able to cover most of the cases, but it is also
very easy to write new expectations.

Below is another example of an Outpost, based on the integration tests:

require 'outpost'
require 'outpost/scouts'
 
class ExamplePingAndHttp < Outpost::DSL
 using Outpost::Scouts::Http => 'master http server' do
   options :host => 'localhost', :port => 9595, :path => '/'
   report :up, :response_body => {:match => /Up/}
 end
 
 using Outpost::Scouts::Ping => 'load balancer' do
   options :host => 'localhost'
   report :up, :response_time => {:less_than => 500}
 end
end

For more details, please check the project’s README, on GitHub.

The future

My plans for Outpost are: SSH support, so you can connect to servers that do
not expose their services to the outer world, a very simple web dashboard to show
the results, better error reporting (for now you can only know which Scout
failed, but not why), a script that automatically generates Outposts for Rails
apps, and the list goes on!

Crazy idea: wouldn’t it be nice to have a script that reads Outposts and
tells if a machine is ready to be in production? TDD for machines? Woot!

Thanks

This project was my personal project for RMU (Ruby Mendicant University) and
so I would like to thank all the people involved for the amazing experience and
reviews on the project. Also thanks to my friends for listening me nagging
about this project for a while. Finally, thanks to Tomás D’Stefano, who
did some work on my previous attempt to write Outpost, your work won’t be
thrown away.

Closing

Please share your thoughts about this project. I would love to know your opinion!

Since moving to Mac, I always find it surprising that the default Mac OS X Ruby and IRB doesn’t allow inputs with accented characters. So, you cannot do

name = "George Guimarães"

This is annoying for brazilians and anyone who uses non-ASCII characters. The problem is that the default Ruby in Mac OS X isn’t linked against readline. A simple solution is to compile readline on your system, and then compile your own ruby binaries. A better one is to use tools that automate this process.

This post by Christopher Sexton uses homebrew to compile readline-0.6 but compiles ruby by hand. Since we love RVM, we’ll use it too.

If you use Homebrew (and you *should*), just do

brew install readline
brew link readline

Beware: linking readline into your system may break other tools that depends on readline source to compile. It was harmless on my system.

Ok, so now we want to build a new ruby and irb binaries. I wanted to use ruby 1.8.7-p248, so:

rvm install 1.8.7-p248 -C --enable-shared,--with-readline-dir=/usr/local

This tells the configure script to enable shared library linking (it is the default), and to search for readline in /usr/local (homebrew has just linked readline there). You may have to use --force if you already have this ruby version compiled. Newer versions of RVM do not need --force.

Now you can use accented and unicode characters on keyboard input in IRB using ruby 1.8.7.

And you? Do you have any tricks with IRB that you may want to share? Do you use Wirble, Utility Belt or others?

One of the beauties in the Open Source world is the possibility of reading other people source code and learn new things. However, lately I found out that not only the library code, but the test suite of several open source projects are full lessons for us.

In this post, I want to tell you which are the three test suites that I admire the most and why.

Integration award: Railties

Rails 3 has several improvements and not all of them may be visible to the application developer. One of the hidden unicorns is Railties test suite. As Yehuda stated in a blog post during the refactoring of version 2.3 to 3.0:

“Although the Rails initializer tests covered a fair amount of area, successfully getting the tests to pass did not guarantee that Rails booted.”

This happened because, in order to have fast tests, Rails 2.3 suite stubbed and mocked a significant part of the booting process. The new test suite is able to create a new application using the application generator, change configuration options, add plugins and engines, boot it and even make HTTP requests using Rack::Test.

For instance, take a look at this test which ensures that app/metals inside plugins are successfully added to the application middleware stack:

def test_plugin_metals_added_to_middleware_stack
  @plugin.write 'app/metal/foo_metal.rb', <<-RUBY
    class FooMetal
      def self.call(env)
        [200, { "Content-Type" => "text/html"}, ["FooMetal"]]
      end
    end
  RUBY
 
  boot_rails
  require 'rack/test'
  extend Rack::Test::Methods
 
  get "/not/slash"
  assert_equal 200, last_response.status
  assert_equal "FooMetal", last_response.body
end

The most important lesson here is: whenever mocking or stubbing in our tests, we still need to add tests without the mocks and stubs to ensure all API contracts are respected.

Readability award: Capybara

Capybara is a tool to aid writing acceptance tests for web applications. Capybara can use several drivers to interact with a web application, as Selenium, Celerity or even Rack::Test. Each driver needs a different setup and has different features. For instance, both Selenium and Celerity can handle javascript, but not Rack::Test.

As you may imagine, all these different drivers can make a test suite become a real spaghetti. However, Jonas Nicklas was able to transform a potential problem into a very elegant and readable test suite with Rspec help. Here is, for instance, the tests for selenium:

describe Capybara::Driver::Selenium do
  before do
    @driver = Capybara::Driver::Selenium.new(TestApp)
  end
 
  it_should_behave_like "driver"
  it_should_behave_like "driver with javascript support"
end

Each behavior group above (“driver” and “driver with javascript support”) is inside Capybara library allowing everyone to develop its own extensions using a shared suite. For instance, if a driver has javascript support, it means the following tests should pass:

shared_examples_for "driver with javascript support" do
  before { @driver.visit('/with_js') }
 
  describe '#find' do
    it "should find dynamically changed nodes" do
      @driver.find('//p').first.text.should == 'I changed it'
    end
  end
 
  describe '#drag_to' do
    it "should drag and drop an object" do
      draggable = @driver.find('//div[@id="drag"]').first
      droppable = @driver.find('//div[@id="drop"]').first
      draggable.drag_to(droppable)
      @driver.find('//div[contains(., "Dropped!")]').should_not be_nil
    end
  end
 
  describe "#evaluate_script" do
    it "should return the value of the executed script" do
      @driver.evaluate_script('1+1').should == 2
    end
  end
end

Capybara test suite is one of the best examples of using tests as documentation. By skimming the test suite you can easily know which features are supported by each driver! Sweet, isn’t it?

Friendliness award: I18n

When you are a big Open Source project, your test suite needs to be easy to run in order to new developers can create patches without hassle. The I18n library for Ruby definitely meets the big Open Source project requirement since it’s widely used and provides several extensions.

However, some of these extensions depends on ActiveRecord, some in ruby2ruby, others in ruby-cldr… and soon it will even support a few Key-Value stores, as Tokyo and Redis. Due to all these dependencies, you would probably imagine that running I18n test suite would require several trials and a lot of configuration before it finally works, right?

WRONG! If you don’t have ActiveRecord, I18n will say: “hey, you don’t have ActiveRecord” but still run the part of test suite that does not depend on it. So if a developer wants to fix or add something trivial, he doesn’t need to worry with installing all sorts of dependencies.

Besides, as mentioned a couple months ago, the I18n library allows you to create several combinations of backends. In other words, the I18n test suite needs to ensure that all these different combinations work as expected.

This problem is quite similar to the one in Capybara which needs to test different drivers. However, I18n uses Test::Unit thus it cannot use shared examples groups as in Rspec. So how were I18n developers able to solve this issue? Using Ruby modules!

Here are the tests for the upcoming KeyValue backend:

require 'test_helper'
require 'api'
 
class I18nKeyValueApiTest < Test::Unit::TestCase
  include Tests::Api::Basics
  include Tests::Api::Defaults
  include Tests::Api::Interpolation
  include Tests::Api::Link
  include Tests::Api::Lookup
  include Tests::Api::Pluralization
  # include Tests::Api::Procs
  include Tests::Api::Localization::Date
  include Tests::Api::Localization::DateTime
  include Tests::Api::Localization::Time
  # include Tests::Api::Localization::Procs
 
  STORE = Rufus::Tokyo::Cabinet.new('*')
 
  def setup
    I18n.backend = I18n::Backend::KeyValue.new(STORE)
    super
  end
 
  test "make sure we use the KeyValue backend" do
    assert_equal I18n::Backend::KeyValue, I18n.backend.class
  end
end

Each included module above adds a series of tests to the backend. Since key-value backends cannot store procs, we don’t include any test related to procs.

Wrapping up

These three are my favorite test suites and also part of my favorite open source projects!

We’ve adopted Capybara as the official testing tool at PlataformaTec for some time already and I18n is one of the subjects of my upcoming book about Rails 3. In one specific chapter, we will build a tool that stores I18n translations into TokyoCabinet, which allows us to create and update translations through a web interface, similarly to ActiveRecord. The only difference is that TokyoCabinet is waaaay faster.

Finally, the fact you can mimic several of Rspec features using simple Ruby (like Capybara using shared example groups and I18n simply using modules) will be part of my talk in Euruko 2010 entitled DSL or NoDSL: The power is in the middle. The talk will show cases where DSLs mimics much of the behavior provided by Ruby and discuss what we are winning and/or losing in such cases.

Keep following us and, until the next blog post is out, we would love to hear in the comments which are your favorite test suites!