Setting up a machine for development is always a manual, annoying task. Here at Plataformatec we have a lot of projects that have different software dependencies, so every time one of our members has to start working on a new project, he needs to setup his machine to meet the project’s requirements, just like every team member of that project already did.

As such, we always end up with some questions: Is this project using MySQL, PostgreSQL or MongoDB? Do I need to install a JavaScript runtime? What about QT? Which Ruby version do I need?

Every team member joining an existing project gets stuck on those same questions, while all they want is to do what they do best, write software.

The problem is even worse when the person is a newcomer, as he may not have any previous knowledge about which software we use.

So, to help our team mates, the newcomers and to make easier and faster to get things done, we started to automate this setup, initially working in our own tool until the day GitHub released Boxen.

About Boxen

Boxen is a tool for automating your machine setup. It works on top of the Puppet project and has some utilities and default modules to make easier to provide and set up services and projects.

Also, Boxen gives you a way to define a personal manifest where you can put personal configurations like you favorite text editor, setting up your dotfiles, or other utility softwares.

How we use Boxen

As a consultancy company we have more than one project running at the same time. Besides that, one of our practices is team rotation, which means that every now and then people move to different teams. Boxen helped us to make those exchanges smoother.

Given that one person has already set up his project to use Boxen, the next person who joins the team will only have to run one command to get his machine ready to work.

The same occurs with problems that someone may have when setting up his machine. Once someone fixes the issue, everyone on the team will take advantage of that fix.

Right now we have some projects set up to use Boxen, and some of them are open source projects like Rails, Simple Form and Elixir.

Our Rails project configuration is open source for those who are interested to contribute to Rails.

Boxen is not only useful for tech people. Every person in the company can use it. Right now we have 10 people using Boxen, where 9 of them are developers and 1 is a project manager.

Conclusion

We have found Boxen to be very useful for our company and we have made it our default tool to set up our machines. Also, we think it is a great way to spread and perpetuate knowledge across your team about the project’s infrastructure and configuration.

I, in particular, learned a lot about systems operation and configuration in the path of making Boxen part of our toolset. Also, I’m very happy to say that I have been granted commit access to the Boxen organization and we will help the GitHub guys and the Boxen team to move this project forward.

And you, what is your team using to set up your machines?

When people talk about mirroring a git repository, usually we have a simple answer in mind:

Just git clone the repo and you’re set!!

However, what we want with mirroring is to replicate the state of an origin repository (or upstream repository). By state, we mean all the branches (including master) and all the tags as well.

You’ll need to do this when migrating your upstream repository to a new “home”, like when switching services like GitHub.

As with most tools, there’s a lot of ways to accomplish that, but I’ll be focusing on two of them. The difference lays on whether you already have a working copy of that repository or not.

Mirroring a git repository without a local copy

If you haven’t cloned the repository before, you can mirror it to a new home by

$ git clone --mirror git@example.com/upstream-repository.git
$ cd upstream-repository.git
$ git push --mirror git@example.com/new-location.git

This will get all the branches and tags that are available in the upstream repository and will replicate those into the new location.

Warning

Don’t use git push --mirror in repositories that weren’t cloned by --mirror as well. It’ll overwrite the remote repository with your local references (and your local branches). This is not what we want. Read the next section to discover what to do in these cases.

Also git clone --mirror is prefered over git clone --bare because the former also clones git notes and some other attributes.

Mirroring a git repository if you already have a local working copy

By working copy, we mean a “normal” repository, in which you have the files that are being tracked into git and where you perform commands like git add and so on.

In this case, you may have a lot of local branches and tags that you don’t want to copy to the new location. But you do have references to remote branches. You can view them with git branches -r. If you pay attention to that list, tough, you may notice that you have a lot of branches that were already deleted in the upstream repository. Why?

Cleaning old references to remote branches

By default, when you do a git fetch or git pull, git will not delete the references to branches that were deleted in the upstream repository (you may view them in your .git/refs/remotes dir). We need to clean those old references before mirroring them to a new location.

To do so, run

$ git fetch --prune

This will update your references to the origin repository and also clean the stale branches reported by git branch -r.

Finally, mirroring the repository to a new location

Now we’re ready to send those updated references back to the origin repository:

$ git push --prune git@example.com:/new-location.git +refs/remotes/origin/*:refs/heads/* +refs/tags/*:refs/tags/*

Ok, what just happened here?!

We want those references inside the .git/refs/remotes/origin to be the LOCAL references in the new location. The local references there will be stored in the refs/heads dir. Same thing happens to tags.

The + sign indicates that we want to overwrite any reference there may already exist.

--prune means we want to delete any reference that may exist there if we don’t have such reference in our refs/remotes/origin/* (and tags) references.

Conclusion

Git is certainly not an easy tool to learn. Although when you do, it turns into a very powerful and flexible tool.

If you want to learn more about it, please see the excelent book written by Scott Chacon and available for free.

What about you? Have any tips on git you want to share?

This week we released the first release candidate version of Devise that is fully compatible with Rails 4, and we’re bumping its version to 3.0. This version completely drops support for Rails 3.1 and Ruby 1.8.7, only keeping compatibility with both Rails 3.2 and Rails 4, running with Ruby 1.9.3 and 2.0.

This rc version took some time to get ready, we’ve been running a rails4 branch for some time already and one of the reasons was because of the changes required to make it compatible with the new strong parameters API from Rails 4. We are aware that some people have been using this branch since Rails 4.0 beta1 with success, and we’re now inviting you to try 3.0 rc with the recent release of Rails 4.0 rc1.

Devise stable

Together with the 3.0 beta version, we’ve released Devise 2.2.4 with a few enhancements and bug fixes, make sure to check the changelog to see the new goodies. All changes are also included in the rc version.

Simple Form

Simple Form has been running a 3.0 rc version for a couple months already, fully compatible with Rails 4 as well, and today we are releasing its release candidate version. In Simple Form master we just dropped support to the 3.x Rails series, focusing our work on Rails 4 compatibility from now on, due to a series of improvements in Rails 4 regarding form helpers – but don’t worry, we will be keeping a v2.1 branch with Rails 3.2 compatibility for a while.

We have some cool plans to improve the wrappers API even further, but that’s subject for another blog post :).

Responders

Responders has been around for quite some time already and we use it in most of our projects, so today we’re celebrating its 1.0 release candidate version, specially to support Rails 4.

Show For

Show For just got a new stable release, v0.2.6, with all the enhancements and bug fixes that were in master, plus a v0.3.0 rc version that adds Rails 4 support.

Mail Form

Mail Form also got a new 1.5 rc release with Rails 4.0 compatibility. Nothing else has changed from the current 1.4 version.

Has Scope

Has Scope is getting a new 0.6 rc version with Rails 4.0 compatibility, including a couple of fixes that were already present in master.

Compatibility

All these new releases are officially dropping support to Rails 3.0 and 3.1, and Ruby 1.8.7. We’ll keep compatibility with Rails 3.2 and 4.0 from now on, all of them on the same branches except for Simple Form which has different branches for each Rails version.

Wrapping up

We’ve got new hot releases for you to try out with Rails 4, please give them a try and let us know if you find any issue or have any feedback.

We’d also like to specially thank everyone involved in helping us getting these projects up and running in Rails 4, without you folks that’d have never been possible.

Enjoy <3

We are very glad to announce the logos for two of our favorite Rails open source projects…

Simple Form:

Simple Form Logo

And Devise:

Devise Logo

We would like to congratulate our designer, Bruna Kochi, who was able to capture the essence of each project in their logos. We will write about their design process soon!

Those projects have been in the Rails community for almost 4 years and it was about time for them to have their own visual identity! We would like to thank all contributors and users who have helped those projects to be more robust, flexible and popular!

We have a gem available for every kind of feature or scenario we might face in our applications and that may help us focus our development time on things that are more important to our applications. But, every now and then, these packaged solutions aren’t exactly what we need, and some sort of customization needs to be done on top of that – a different authentication strategy, new ways to query for data and several different things that our business rules might require.

So, we jump on top of the existing code to bend it to our needs but sometimes things can go south and we end up in a mess of hacks, unstable code and bad experiences. After some time, we started to develop a few guidelines of our own to avoid the mistakes of the past and look forward to write better applications. These are some of the ideas that I follow to avoid complications when dealing with 3rd party code:

Don’t fear the source

The source code and its documentation are your best friends on this. Having a local clone of a dependency repository lets you ack/grep it inside out to see how the code is structured to identify the good and bad parts to mess with. You can test your changes against its test suite to see if you might break something or not and that’s already one step closer to contribute back to the project.

Respect method visibility

Method visibility is an important tool to ensure that you aren’t messing with the wrong pieces of code from a gem. Public and protected methods are meant to be overriden when necessary, but private ones aren’t. They are usually doing the work that you don’t want the trouble to do it yourself, and maybe that’s why you are using the dependency after all.

For example, ActiveRecord adds a lot of private methods to handle the persistence of your models that you shouldn’t mess with, but the public API is stable enough for you to use it for whatever you need.

Monkey patch at your own peril

Ruby lets you monkey patch everything but that doesn’t mean you should. While this might make a lot of sense for libraries that extend the Ruby stdlib (like ActiveSupport), monkey patching someone else constant might bite you back later. Overusing monkey patches might be a serious block when updating your application to newer versions of a big dependency of your project (for example, Rails).

When you monkey patch, you are usually messing with a very internal piece of a component that might be far from it’s public API. So, you can’t predict how that class or module will behave when a new version is released or what other parts of the code are using that internal API. Classes get renamed and refactored everyday, and it’s hard to ensure your patches will keep up with those changes.

Composition (and inheritance) as extension mechanisms

A lot of gems provide a series of configuration options that you can drop in an initializer and get the behavior you need, or maybe a specific configuration might be missing. You might feel the urge to send a pull request adding a new configuration to the project, but hold that idea for a second. Can’t you do it by overriding a method or using a custom component of your own?

Inheritance and composition can be a better choice for a lot of customizations since they are easier to test and to isolate the effects on your application. While a configuration setting is global and affects your entire application, an isolated change will have a much smaller impact on your code.

Take for instance the to_param and to_partial_path methods from ActiveModel. You can override them in your models to change how your views will interact with them, and that goes in a per model basis, since you usually won’t do that for your entire application. Imagine if you need to change a configuration instead overriding a method: You would have to do something weird like this:

# A regular configuration inside an initializer
config.action_view.parameterize_method = :slug
 
# But what if I need a per model configuration? Well, use a Hash!
config.action_view.parameterize_methods = { post: :slug, user: :id }

While just overriding the to_param method in your Post model is a lot easier than this.

Another example of composition I came across recently was the tokenizer option on the LengthValidator. Given that you have a description column in your database that accepts HTML tags like strong and em, and you want to validate the length of the text, but not the HTML, you can provide an object that responds to call and strips away the HTML from the string, so the validation will be executed against the raw text instead of the whole HTML of it.

class MyOwnTokenizer
  def call(text)
    # do whatever you need with `text`.
  end
end
 
# on your model…
validates :description, :length { tokenizer: MyOwnTokenizer.new }

Your code, your problem

Remember to test your changes. Once you change a default behavior or tweak some specific configuration that might have side effects on other parts of your application, your test coverage will help ensure that this behavior won’t break once you update a dependency on your project.

You usually shouldn’t worry about testing library defaults (like testing the validations on your models that you configured with ActiveModel validation methods), but once you customize something, that piece of code is your responsibility.

So, if you added your own tokenizer use along with a LengthValidator on your application, be sure to write at least an unit test for it to ensure that it works as expected.

Contribute back

Sometimes you might notice (or need) an improvement to a library that won’t change anything on its public API but will make your life easier when extending it. You can’t expect that the maintainers will discover every spot that can or might be overriden, so it’s important to bring your experience on using it to the table and help others. You can extract a specific behavior to an isolated component, or improve some internal logic so it might be easier to extend it in the future. There’s nothing but love for such kind of contribution.

A while ago this pull request changed how Rails added the associations proxies to a model that is using belongs_to and friends. While it didn’t changes a single bit about the public API for the associations, it changed how you can extend them to add your specific behavior.

Wrapping Up

These steps might not fit everyone’s workflow, but we need to keep in mind that dealing with external dependencies requires a thoughtful approach to avoid the results being harmful to your projects.

And what about you, my fellow developer: how do you approach the need for something more than a gem’s default behavior? Jump on our comments thread to discuss more about it.

Here at Plataformatec we use Github Pull Requests a lot for code review and this usually yields tons of constructive comments and excellent discussions from time to time. One of the recent topics was about whether we should use scopes or class methods throughout the project to be consistent. It’s also not hard to find discussions about it all over the internet. The classic comment usually boils down to “there is no difference between them” or “it is a matter of taste”. I tend to agree with both sentences, but I’d like to show some slight differences that exist between both.

Defining a scope

First of all, lets get a better understanding about how scopes are used. In Rails 3 you can define a scope in two ways:

class Post < ActiveRecord::Base
  scope :published, where(status: 'published')
  scope :draft, -> { where(status: 'draft') } 
end

The main difference between both usages is that the :published condition is evaluated when the class is first loaded, whereas the :draft one is lazy evaluated when it is called. Because of that, in Rails 4 the first way is going to be deprecated which means you will always need to declare scopes with a callable object as argument. This is to avoid issues when trying to declare a scope with some sort of Time argument:

class Post < ActiveRecord::Base
  scope :published_last_week, where('published_at >= ?', 1.week.ago)
end

Because this won’t work as expected: 1.week.ago will be evaluated when the class is loaded, not every time the scope is called.

Scopes are just class methods

Internally Active Record converts a scope into a class method. Conceptually, its simplified implementation in Rails master looks something like this:

def self.scope(name, body)
  singleton_class.send(:define_method, name, &body)
end

Which ends up as a class method with the given name and body, like this:

def self.published
  where(status: 'published')
end

And I think that’s why most people think: “Why should I use a scope if it is just syntax sugar for a class method?”. So here are some interesting examples for you to think about.

Scopes are always chainable

Lets use the following scenario: users will be able to filter posts by statuses, ordering by most recent updated ones. Simple enough, lets write scopes for that:

class Post < ActiveRecord::Base
  scope :by_status, -> status { where(status: status) }
  scope :recent, -> { order("posts.updated_at DESC") }
end

And we can call them freely like this:

Post.by_status('published').recent
# SELECT "posts".* FROM "posts" WHERE "posts"."status" = 'published' 
#   ORDER BY posts.updated_at DESC

Or with a user provided param:

Post.by_status(params[:status]).recent
# SELECT "posts".* FROM "posts" WHERE "posts"."status" = 'published' 
#   ORDER BY posts.updated_at DESC

So far, so good. Now lets move them to class methods, just for the sake of comparing:

class Post < ActiveRecord::Base
  def self.by_status(status)
    where(status: status)
  end
 
  def self.recent
    order("posts.updated_at DESC")
  end
end

Besides using a few extra lines, no big improvements. But now what happens if the :status parameter is nil or blank?

Post.by_status(nil).recent
# SELECT "posts".* FROM "posts" WHERE "posts"."status" IS NULL 
#   ORDER BY posts.updated_at DESC
 
Post.by_status('').recent
# SELECT "posts".* FROM "posts" WHERE "posts"."status" = '' 
#   ORDER BY posts.updated_at DESC

Oooops, I don’t think we wanted to allow these queries, did we? With scopes, we can easily fix that by adding a presence condition to our scope:

scope :by_status, -> status { where(status: status) if status.present? }

There we go:

Post.by_status(nil).recent
# SELECT "posts".* FROM "posts" ORDER BY posts.updated_at DESC
 
Post.by_status('').recent
# SELECT "posts".* FROM "posts" ORDER BY posts.updated_at DESC

Awesome. Now lets try to do the same with our beloved class method:

class Post < ActiveRecord::Base
  def self.by_status(status)
    where(status: status) if status.present?
  end
end

Running this:

Post.by_status('').recent
NoMethodError: undefined method `recent' for nil:NilClass

And :bomb:. The difference is that a scope will always return a relation, whereas our simple class method implementation will not. The class method should look like this instead:

def self.by_status(status)
  if status.present?
    where(status: status)
  else
    all
  end
end

Notice that I’m returning all for the nil/blank case, which in Rails 4 returns a relation (it previously returned the Array of items from the database). In Rails 3.2.x, you should use scoped there instead. And there we go:

Post.by_status('').recent
# SELECT "posts".* FROM "posts" ORDER BY posts.updated_at DESC

So the advice here is: never return nil from a class method that should work like a scope, otherwise you’re breaking the chainability condition implied by scopes, that always return a relation.

Scopes are extensible

Lets get pagination as our next example and I’m going to use the kaminari gem as basis. The most important thing you need to do when paginating a collection is to tell which page you want to fetch:

Post.page(2)

After doing that you might want to say how many records per page you want:

Post.page(2).per(15)

And you may to know the total number of pages, or whether you are in the first or last page:

posts = Post.page(2)
posts.total_pages # => 2
posts.first_page? # => false
posts.last_page?  # => true

This all makes sense when we call things in this order, but it doesn’t make any sense to call these methods in a collection that is not paginated, does it? When you write scopes, you can add specific extensions that will only be available in your object if that scope is called. In case of kaminari, it only adds the page scope to your Active Record models, and relies on the scope extensions feature to add all other functionality when page is called. Conceptually, the code would look like this:

scope :page, -> num { # some limit + offset logic here for pagination } do
  def per(num)
    # more logic here
  end
 
  def total_pages
    # some more here
  end
 
  def first_page?
    # and a bit more
  end
 
  def last_page?
    # and so on
  end
end

Scope extensions is a powerful and flexible technique to have in our toolchain. But of course, we can always go wild and get all that with class methods too:

def self.page(num)
  scope = # some limit + offset logic here for pagination
  scope.extend PaginationExtensions
  scope
end
 
module PaginationExtensions
  def per(num)
    # more logic here
  end
 
  def total_pages
    # some more here
  end
 
  def first_page?
    # and a bit more
  end
 
  def last_page?
    # and so on
  end
end

It is a bit more verbose than using a scope, but it yields the same results. And the advice here is: pick what works better for you but make sure you know what the framework provides before reinventing the wheel.

Wrapping up

I personally tend to use scopes when the logic is very small, for simple where/order clauses, and class methods when it involves a bit more complexity, but whether it receives an argument or not doesn’t really matter much to me. I also tend to rely more on scopes when doing extensions like showed here, since it’s a feature that Active Record already gives us for free.

I think it’s important to clarify the main differences between scopes and class methods, so that you can pick the right tool for the job™, or the tool that makes you more comfortable. Whether you use one or another, I don’t think it really matters, as long as you write them clear and consistently throughout your application.

Do you have any thought about using scopes vs class methods? Make sure to leave a comment below telling us what you think, we’d love to hear.