{"id":714,"date":"2010-02-10T15:47:33","date_gmt":"2010-02-10T17:47:33","guid":{"rendered":"http:\/\/blog.plataformatec.com.br\/?p=714"},"modified":"2010-04-16T16:00:15","modified_gmt":"2010-04-16T19:00:15","slug":"monitoring-delayed-job-with-bluepill-and-capistrano","status":"publish","type":"post","link":"https:\/\/blog.plataformatec.com.br\/2010\/02\/monitoring-delayed-job-with-bluepill-and-capistrano\/","title":{"rendered":"Monitoring Delayed Job with Bluepill and Capistrano"},"content":{"rendered":"

So, you already did the right choice of using Delayed Job for your background processing, great! But, how are you going to be certain that your background processing will still be happening while you are sleeping? And if your Delayed Job process goes down, are you going to wake up in the dawn e restart it manually? I wouldn’t do that, I really appreciate my sleep. So, what’s the solution?<\/p>\n

As rubyists and railers, we already know there are solutions, like God<\/a>, that do this job for us. However, there are another solutions, like Bluepill<\/a>. Bluepill is a process monitoring tool like God, but, unlike God, it doesn’t have memory leak<\/a>, according to its authors.<\/p>\n

Well, as I don’t want to wake up in the dawn to restart my Delayed Job process, and neither I want to restart my God process because of memory leaking, I decided to use Bluepill. But, how I use Bluepill to monitor my Delayed Job process?<\/p>\n

In order to configure Bluepill to monitor Delayed Job, and use Capistrano to automate some tasks, we have basically 4 steps:<\/p>\n

    \n
  1. Install and configure Delayed Job<\/li>\n
  2. Install and configure Bluepill<\/li>\n
  3. Write a Capistrano Recipe for Bluepill<\/li>\n
  4. Set some stuff in \/etc\/sudoers<\/li>\n<\/ol>\n

    Let’s take a look at each step.<\/p>\n

    Installing and configuring Delayed Job<\/h3>\n

    Installing and configuring Delayed Job is super simple, just read the project’s README<\/a>, which is very clear. I also recommend watching the RailsCast episode<\/a> about Delayed Job, there is some good information there, like using the Delayed Job CollectiveIdea’s fork, instead of the original repo’s version.<\/p>\n

    Installing and configuring Bluepill<\/h3>\n

    There’s no secret in installing Bluepill too, you just need to read the project’s README<\/a> and follow its steps.<\/p>\n

    In the confiuration part, you can see all the options in the README<\/a> too. In my case, the configuration file is at RAILS_ROOT\/config\/production.pill<\/code>.<\/p>\n

    Bluepill.application(\"my_app\") do |app|\r\n  app.process(\"delayed_job\") do |process|\r\n    process.working_dir = \"\/home\/deploy\/my_app\/current\"\r\n\r\n    process.start_grace_time    = 10.seconds\r\n    process.stop_grace_time     = 10.seconds\r\n    process.restart_grace_time  = 10.seconds\r\n\r\n    process.start_command = \"ruby script\/delayed_job -e production start\"\r\n    process.stop_command  = \"ruby script\/delayed_job -e production stop\"\r\n\r\n    process.pid_file = \"\/home\/deploy\/my_app\/shared\/pids\/delayed_job.pid\"\r\n    process.uid = \"deploy\"\r\n    process.gid = \"deploy\"\r\n  end\r\nend\r\n<\/pre>\n

    However, I had a little problem between the interaction of Bluepill and Delayed Job. Delayed Job is not interpreting very well the -e production<\/code> flag. You can see more details about that in an issue<\/a> I opened.<\/p>\n

    The first solution I thought was to use RAILS_ENV=production ruby script\/delayed_job start<\/code>, however, for some reason that I don’t know exactly, it didn’t work.<\/p>\n

    So, the solution I came up with was to modify the file in RAILS_ROOT\/script\/delayed_job<\/code> to the following one:<\/p>\n

    \r\n#!\/usr\/bin\/env ruby\r\n\r\n# TODO improve the line of code below\r\n# The line below is just a hack while we wait the delayed job guys answer the following issue\r\n# http:\/\/github.com\/collectiveidea\/delayed_job\/issues#issue\/38\r\nENV['RAILS_ENV'] ||= \"production\"\r\nrequire File.expand_path(File.join(File.dirname(__FILE__), '..', 'config', 'environment'))\r\nrequire 'delayed\/command'\r\nDelayed::Command.new(ARGV).daemonize\r\n<\/pre>\n

    As the -e production<\/code> was not being interpreted properly by Delayed Job, I’m setting manually the RAILS_ENV<\/code> to production<\/code> (line 6), therefore I’m supposing that you are using Bluepill to monitor processes that are in a production environment.<\/p>\n

    Capistrano recipe for Bluepill<\/h3>\n

    In order to automate the start and stop Bluepill’s tasks, I wrote the following capistrano’s recipe:<\/p>\n

    # deploy credentials\r\nset :user, \"deploy\"\r\nset :deploy_to, \"\/home\/deploy\/#{application}\"\r\nset :use_sudo, false\r\nset :rails_env, \"production\"\r\n\r\n# Bluepill related tasks\r\nafter \"deploy:update\", \"bluepill:quit\", \"bluepill:start\"\r\nnamespace :bluepill do\r\n  desc \"Stop processes that bluepill is monitoring and quit bluepill\"\r\n  task :quit, :roles => [:app] do\r\n    sudo \"bluepill stop\"\r\n    sudo \"bluepill quit\"\r\n  end\r\n\r\n  desc \"Load bluepill configuration and start it\"\r\n  task :start, :roles => [:app] do\r\n    sudo \"bluepill load \/home\/deploy\/my_app\/current\/config\/production.pill\"\r\n  end\r\n\r\n  desc \"Prints bluepills monitored processes statuses\"\r\n  task :status, :roles => [:app] do\r\n    sudo \"bluepill status\"\r\n  end\r\nend\r\n<\/pre>\n

    Note that instead of using the run<\/code> capistrano’s method, I’m using the sudo<\/code> method. I’m doing this because the bluepill command must be run as root. And that, takes us to the next topic.<\/p>\n

    \/etc\/sudoers<\/h3>\n

    Since I need to run the bluepill command as root, should I change from set :user, \"deploy\"<\/code> to set :user, \"root\"<\/code>? I think it’s not a good idea, we don’t like to give root access to anything, even to deployment. So, what should I do? It’s simple, you just need to edit your sudoers file.<\/p>\n

    In order to do this, you need to use the visudo<\/code> command to open and edit the \/etc\/sudoers<\/code> file. Once with the file opened, just add the following line to the end of the file:<\/p>\n

    deploy ALL=(ALL) NOPASSWD: \/usr\/local\/bin\/bluepill<\/code><\/p>\n

    Now, you’re done, the deploy user already can do sudo bluepill<\/code> without giving any password. Problem solved without opening security holes.<\/p>\n

    After these 4 steps, you’re ready to sleep at night without worrying about your Delayed Job process. And, if you want to know the status of the monitored processes by Buepill, you just need to run the following capistrano task in your local machine:<\/p>\n

    cap bluepill:status<\/code><\/p>\n

    And you, what are your solutions to sleep in peace?<\/p>\n

    Update:<\/strong> if are having trouble with restarting Bluepill with Capistrano, take a look at this<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"

    So, you already did the right choice of using Delayed Job for your background processing, great! But, how are you going to be certain that your background processing will still be happening while you are sleeping? And if your Delayed Job process goes down, are you going to wake up in the dawn e restart … \u00bb<\/a><\/p>\n","protected":false},"author":5,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"ngg_post_thumbnail":0,"footnotes":""},"categories":[1],"tags":[72,71,73,74],"aioseo_notices":[],"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/blog.plataformatec.com.br\/wp-json\/wp\/v2\/posts\/714"}],"collection":[{"href":"https:\/\/blog.plataformatec.com.br\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.plataformatec.com.br\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.plataformatec.com.br\/wp-json\/wp\/v2\/users\/5"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.plataformatec.com.br\/wp-json\/wp\/v2\/comments?post=714"}],"version-history":[{"count":26,"href":"https:\/\/blog.plataformatec.com.br\/wp-json\/wp\/v2\/posts\/714\/revisions"}],"predecessor-version":[{"id":877,"href":"https:\/\/blog.plataformatec.com.br\/wp-json\/wp\/v2\/posts\/714\/revisions\/877"}],"wp:attachment":[{"href":"https:\/\/blog.plataformatec.com.br\/wp-json\/wp\/v2\/media?parent=714"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.plataformatec.com.br\/wp-json\/wp\/v2\/categories?post=714"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.plataformatec.com.br\/wp-json\/wp\/v2\/tags?post=714"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}