organizing your specs into groups and running them separately

A while back I answered this StackOverflow question.

When you have a lot of specs, it makes sense to run them in separate logical groups and not just with rspec spec or something like that.

This way, you can save time and you can also run them in separate processes on the CI.

For example:

bundle exec rake spec:unit
bundle exec rake spec:integration
bundle exec rake spec:api

In order to achieve this, you need to change the spec.rake file.

  namespace :spec do do |t|
      t.pattern = Dir['spec/*/**/*_spec.rb'].reject{ |f| f['/api/v1'] || f['/integration'] }
    end do |t|
      t.pattern = "spec/*/{api/v1}*/**/*_spec.rb"
    end do |t|
      t.pattern = "spec/integration/**/*_spec.rb"

You can continue customizing that all you want, you can run specific specs that are the most important to you.

I find those groups useful for most of my use cases, but with minor changes you can make it fit yours

Using Rspec Tags

You can use tags for that as well, but I find that more tedious and you can forget to tag something.

For example:

  it "should do some integration test", :integration => true do
   # something

searching through the history of commands you ran in terminal

Whenever I am logged in to a server or even when I am working on my own machine, I keep searching through the command history through up and down arrows.

While this can be efficient if you have 2-3 commands, it can be quite frustrating to find specific commands.

That is something I keep doing over and over again, and now I have a better way, I just grep through the list of commands, find the one I want, copy it and paste it into a new command, and I’m done.

This saves me a lot of time.

Here’s how:

To show the history of commands you just do:


You probably know the rest, but you can just pipe the history into grep and search your history

history | grep {command_or_part_of_command}

For example:

history | grep cp -R


Making Nokogiri install again on Mountain Lion

As probably any geek out there, I upgraded my OS to Apple Mountain Lion.

The upgrade created a lot of problems for me, I basically had to reinstall almost everything, from MySql to homebrew.

I am not sure if everyone experienced the same thing, but that was the case for me.

One of the problems I encountered was that I could not install Nokogiri anymore on my machine, bundler would not install it and complain about dependencies not being installed (specifically libxml)

To fix it, you need to reinstall Ruby using RVM with livxml properly linked.

First, install libxml and libxslt through homebrew, like so:

brew install libxml2 libxslt
brew link libxml2 libxslt

If that doesn’t work you probably need to install libiconv like so:

mkdir temp
cd temp
tar xvfz libiconv-1.13.1.tar.gz
cd libiconv-1.13.1
./configure --prefix=/usr/local/Cellar/libiconv/1.13.1
sudo make install

And then install libxml and libxslt again

brew install libxml2 libxslt
brew link libxml2 libxslt

Once that’s done without errors, reinstall Ruby.

rvm reinstall ruby-1.9.3-p194

RVM will figure out those libraries location and properly install Ruby with those linked up.


Creating self starting worker machines for fun and profit

Lately, I needed to create worker machines (Machines that have Resque workers on them).

Usually, the process is very manual and tedious, you need to deploy the code to each machine (on code change), you need to start/stop/restart the workers and more.

Since I needed A LOT of workers, I really wanted to make the process automatic.

I wanted the machine to be in charge of everything it needs.


  • Pull latest git code
  • Copy file to the current directory
  • Startup god (which will start the Resque processes)

I wanted the machine to be in charge of all this when it boots up, so in case I need to update the code or something, all I need is to reboot the machines and that’s it.

Also, scaling up the process is super easy, all I need is to add more machines.

The logic behind it was also being able to use spot-instances on Amazon, which are much cheaper.

So, as always, I thought I would document the process and open source it.


What are the challenges we are looking at, when we want to do something like that.

  1. Awareness of RVM, Ruby version
  2. Awareness of Bundler and Gem versions.

Those are things we need to consider.

Breaking down the process into pseudo code.

So, let’s break down the process into pseudo code

Go to the temp folder
git pull from origin + branch you want
Copy all files from temp folder to current folder
Go To current folder, bundle install
Start God

RVM Wrappers

For the last 2 parts of the process, if you try to do just bundle install --deployment for example, the script will fail, it does not know what bundle is, it resolves to the default Ruby installed on the machine at this point.

Since want the last 2 processes to go through RVM, we need to create a wrapper script.

Creating RVM Wrappers

You can create RVM wrappers like this:

rvm wrapper {ruby_version@gemset} {prefix} {executable}

For example

rvm wrapper 1.9.2 bootup god

This will generate this file: (called bootup_god)

#!/usr/bin/env bash

if [[ -s "/usr/local/rvm/environments/ruby-1.9.2-p290" ]]
  source "/usr/local/rvm/environments/ruby-1.9.2-p290"
  exec god "$@"
  echo "ERROR: Missing RVM environment file: '/usr/local/rvm/environments/ruby-1.9.2-p290'" >&2
  exit 1

I did the same for bundle which generated this file: (called bootup_bundle)

#!/usr/bin/env bash

if [[ -s "/usr/local/rvm/environments/ruby-1.9.2-p290" ]]
  source "/usr/local/rvm/environments/ruby-1.9.2-p290"
  exec bundle "$@"
  echo "ERROR: Missing RVM environment file: '/usr/local/rvm/environments/ruby-1.9.2-p290'" >&2
  exit 1

The files are created in vim /usr/local/rvm/bin/.

Now that we have the wrappers, we made sure the boot up process will be aware of RVM and we can continue.

RC files

If you want something to run when the machine starts you need to get to know the rc files.

Usually, those files are located in /etc/ folder (may vary of course)

The files are loaded and executed in a specific order


If you want something to run when ALL the rest of the machine bootup process ran, you put it at the end of the rc.local file.

One gotcha here is that the file must end with exit 0, or your script will not run at all, and good luck finding out why :P

I put this line at the end, right before the exit 0

sh /root/


Now that I have the wrappers, I have the line in rc.local I can finalize the process with creating the shell script.

This is what the script looks for me

cd {temp_folder}
git pull origin {branch_name}
cd ..
cp -apRv {source_folder}/* {destination_folder} --reply=yes
echo "Running bundle install"
cd /root/c && /usr/local/rvm/bin/bootup_bundle install --path /mnt/data-store/html/gogobot/shared/bundle --deployment --without development test
echo "Starting RVM"
/usr/local/rvm/bin/bootup_god -c /root/c/god/resque_extra_workers.god

The god file to start the workers is standard.

rails_env   = ENV['RAILS_ENV']  || "production"
rails_root  = ENV['RAILS_ROOT'] || "/mnt/data-store/html/gogobot/current"
WORKER_TIMEOUT = 60 * 10 # 10 minutes

# Stale workers do
  loop do
      `ps -e -o pid,command | grep [r]esque`.split("\n").each do |line|
        parts   = line.split(' ')
        next if parts[-2] != "at"
        started = parts[-1].to_i
        elapsed = -

        if elapsed >= WORKER_TIMEOUT
          ::Process.kill('USR1', parts[0].to_i)
      # don't die because of stupid exceptions
    sleep 30

queue_name = "graph_checkins_import"
num_workers = 10

# FB wall posts
num_workers.times do |num| do |w|
    w.dir      = "#{rails_root}"     = "resque-#{num}-#{queue_name}"    = "resque"
    w.interval = 2.minutes
    w.env      = {"QUEUE"=>"#{queue_name}", "RAILS_ENV"=>rails_env, "PIDFILE" => "#{rails_root}/tmp/resque_#{queue_name}_#{w}.pid"}
    w.pid_file = "#{rails_root}/tmp/resque_#{queue_name}_#{w}.pid"
    w.start    = "cd #{rails_root}/ && bundle exec rake environment resque:work QUEUE=#{queue_name} RAILS_ENV=#{rails_env}"
    w.log      = "#{rails_root}/log/resque_god.log"

    w.uid = 'root'
    w.gid = 'root'

    # restart if memory gets too high
    w.transition(:up, :restart) do |on|
      on.condition(:memory_usage) do |c|
        c.above = 350.megabytes
        c.times = 2

    # determine the state on startup
    w.transition(:init, { true => :up, false => :start }) do |on|
      on.condition(:process_running) do |c|
        c.running = true

    # determine when process has finished starting
    w.transition([:start, :restart], :up) do |on|
      on.condition(:process_running) do |c|
        c.running = true
        c.interval = 5.seconds

      # failsafe
      on.condition(:tries) do |c|
        c.times = 5
        c.transition = :start
        c.interval = 5.seconds

    # start if process is not running
    w.transition(:up, :start) do |on|
      on.condition(:process_running) do |c|
        c.running = false


Now, it’s a completely automatic process, all you need is to run as many machines as you need, once the machine is booted up, it will deploy code to itself, copy files, install gems and run workers.

If you want to restart the process, add machines or anything else, you can do it with the click of a button in the AWS UI.

Comments? Questions?

Please feel free

problem with running spec in vim

I have been using Vim for almost a week now.

I am using the dotvim configuration by Astrails.

All in all, the configuration is awesome and I completely love it, it’s the longest time period I have ever used Vim. Considering that the second place is about 5 minutes, that’s super impressive.

I only have one problem with it.

I took the spec running functions from Gary Bernhardt dotfiles, I just remapped the keys differently.

map <leader>' :call RunTestFile()<cr>
map <leader>; :call RunNearestTest()<cr>

function! RunTestFile(...)
    if a:0
        let command_suffix = a:1
        let command_suffix = ""

    " Run the tests for the previously-marked file.
    let in_test_file = match(expand("%"), '\(.feature\|_spec.rb\)$') != -1
    if in_test_file
        call SetTestFile()
    elseif !exists("t:grb_test_file")
    call RunTests(t:grb_test_file . command_suffix)

function! RunNearestTest()
    let spec_line_number = line('.')
    call RunTestFile(":" . spec_line_number . " -b")

function! SetTestFile()
    " Set the spec file that tests will be run for.
    let t:grb_test_file=@%

function! RunTests(filename)
    " Write the file and run tests for the given filename
    :silent !echo;echo;echo;echo;echo;echo;echo;echo;echo;echo
    :silent !echo;echo;echo;echo;echo;echo;echo;echo;echo;echo
    :silent !echo;echo;echo;echo;echo;echo;echo;echo;echo;echo
    :silent !echo;echo;echo;echo;echo;echo;echo;echo;echo;echo
    :silent !echo;echo;echo;echo;echo;echo;echo;echo;echo;echo
    :silent !echo;echo;echo;echo;echo;echo;echo;echo;echo;echo
    if match(a:filename, '\.feature$') != -1
        exec ":!script/features " . a:filename
        if filereadable("script/test")
            exec ":!script/test " . a:filename
        elseif filereadable("Gemfile")
            exec ":!bundle exec rspec --color " . a:filename
            exec ":!rspec --color " . a:filename

As you can see, I mapped ,; to run line specs, and ,' to run the entire spec file.

The problem is, that the terminal exists immediately after the result, does not wait for me to hit Enter or anything.

You can see the problem demo in this YouTube Video

I asked this question on StackOverflow here

Someone suggested it might be a trailing character, but I checked and there’s nothing like that.

The same person suggested to map just ls like so:

nmap <leader>ls :!ls<cr>

But this command does not exit.

Another thing I tried is this
In a vim session I mapped a key like this

map <Leader>~ :!rspec %<cr>

When I used it, the terminal did not exit as well.

Vim Experts, what am I missing here?

Running specs all the time is something crucial to my workflow, I don’t want to hide vim very time, it breaks my flow.

Any help appreciated.

My entire vim configuration is here: link

copy all files from a folder into a folder without the confirmation on linux

As a developer, there are things that never mind how many times you’ll do them, you will never ever remember how to do it exactly without googling for a couple of minutes or looking through the last executed list of commands on the terminal.

For me, one of those things is how to copy all files and folders from source to destination with automatic reply yes to overwrite requests.

Here’s how you do it

cp -apRv <source_folder>/* <dest_folder>/ --reply=yes


And #notetoself, no need to google anymore, find it on my blog ;)

run rake task in the background and log to file

I am working with rake tasks a lot, I love the simplicity of creating one and just running one on the server.

For the real heavy lifting I am using a queue system of course, but when I just want to throw something in the queue for example, I will usually create a rake task for it and run it on one of the servers.

I use quick and dirty puts messages to log the progress.

For example I could have a rake task like this:

User.find_each do |user|
  puts "Going over user: #{}"

To run this on the server, I just ssh into is and then I do screen so when I log out of the server the session will save the process and not kill it.

Then, I run this command:

rake foo:bar  --trace 2>&1 >> log/some_log_file.log

That’s it, you can now leave the server and let it do all the work, periodically, you can log in and check the progress just by tailing the file.

project specific settings using zshell and dotfiles

I have been working with ZHell pretty much since the first day that I have been using a Mac.

The use I make of to grew with time and definitely the biggest switch was that I moved to using dotfiles for settings in a single location.

I forked Zach Holman dot files here, and I have been adjusting and customizing it ever since.

I am using Ruby/Rails for my everyday work, both for consulting and open source projects, the one thing that I see people do and I don’t like is committing configuration files with password and sensitive information into source control.

When I can, I try to avoid it, so I developed a very easy way to manage my per-project environment variable without going through a huge file.

All of my zsh files are located in a single folder in my dotfiles.

├── aliases.zsh
├── completion.zsh
├── config.zsh
├── functions
│   ├── _boom
│   ├── _brew
│   ├── _c
│   ├── _git-rm
│   ├── _h
│   ├── _rake
│   ├── c
│   ├── gf
│   ├── gitdays
│   ├── h
│   ├── last_modified
│   ├── newtab
│   ├── savepath
│   ├── smartextract
│   ├── start_project
│   ├── verbose_completion
│   └── zgitinit
├── hidden_aliases.zsh
├── projects
│   ├── boto_project.zsh
│   └── octopus_project.zsh
├── prompt.zsh
├── window.zsh
└── zshrc.symlink

As you can see, there’s a special folder there called projects, in which I put a lll of my project specific setting like tokens, passwords and other things.

For example, here’s what a project file might look like:

export OCTOPUS_POSTGRES_PASSWORD=some_password

One thing that is very easy to forget, is that if you open source your dot files (and you should) don’t forget to ignore those files and don’t commit them.

Run Octopress New Post Rake On Zshell

layout: post
title: “run octopress new_post rake on zshell”
date: 2012-07-27 22:53
comments: true
categories: general

This blog is running on Octopress, which is an awesome implementation of a blogging engine in Ruby.

To create a new post, you need to run a rake task like so:

rake new_post["some awesome post title"]

The problem is, that in zsh, those are escaped and you get an error.

I used to do new_post without anything and then edit the file name and the title inside the file BUT there’s actually a better way.

You can either run noglob rake or you can run the rake like this:

rake "new_post[some awesome post title]"