In order to achieve this, you need to change the spec.rake file.
namespace :spec do
Rspec::Core::RakeTask.new(:unit) do |t|
t.pattern = Dir['spec/*/**/*_spec.rb'].reject{ |f| f['/api/v1'] || f['/integration'] }
end
Rspec::Core::RakeTask.new(:api) do |t|
t.pattern = "spec/*/{api/v1}*/**/*_spec.rb"
end
Rspec::Core::RakeTask.new(:integration) do |t|
t.pattern = "spec/integration/**/*_spec.rb"
end
end
You can continue customizing that all you want, you can run specific specs that are the most important to you.
I find those groups useful for most of my use cases, but with minor changes you can make it fit yours
Using Rspec Tags
You can use tags for that as well, but I find that more tedious and you can forget to tag something.
For example:
it "should do some integration test", :integration => true do
# something
end
Whenever I am logged in to a server or even when I am working on my own machine, I keep searching through the command history through up and down arrows.
While this can be efficient if you have 2-3 commands, it can be quite frustrating to find specific commands.
That is something I keep doing over and over again, and now I have a better way, I just grep through the list of commands, find the one I want, copy it and paste it into a new command, and I’m done.
This saves me a lot of time.
Here’s how:
To show the history of commands you just do:
history
You probably know the rest, but you can just pipe the history into grep and search your history
As probably any geek out there, I upgraded my OS to Apple Mountain Lion.
The upgrade created a lot of problems for me, I basically had to reinstall almost everything, from MySql to homebrew.
I am not sure if everyone experienced the same thing, but that was the case for me.
One of the problems I encountered was that I could not install Nokogiri anymore on my machine, bundler would not install it and complain about dependencies not being installed (specifically libxml)
To fix it, you need to reinstall Ruby using RVM with livxml properly linked.
First, install libxml and libxslt through homebrew, like so:
brew install libxml2 libxslt
brew link libxml2 libxslt
If that doesn’t work you probably need to install libiconv like so:
cd
mkdir temp
cd temp
ls
wget http://ftp.gnu.org/pub/gnu/libiconv/libiconv-1.13.1.tar.gz
tar xvfz libiconv-1.13.1.tar.gz
cd libiconv-1.13.1
./configure --prefix=/usr/local/Cellar/libiconv/1.13.1
make
sudo make install
And then install libxml and libxslt again
brew install libxml2 libxslt
brew link libxml2 libxslt
Once that’s done without errors, reinstall Ruby.
rvm reinstall ruby-1.9.3-p194
RVM will figure out those libraries location and properly install Ruby with those linked up.
Lately, I needed to create worker machines (Machines that have Resque workers on them).
Usually, the process is very manual and tedious, you need to deploy the code to each machine (on code change), you need to start/stop/restart the workers and more.
Since I needed A LOT of workers, I really wanted to make the process automatic.
I wanted the machine to be in charge of everything it needs.
Meaning
Pull latest git code
Copy file to the current directory
Startup god (which will start the Resque processes)
I wanted the machine to be in charge of all this when it boots up, so in case I need to update the code or something, all I need is to reboot the machines and that’s it.
Also, scaling up the process is super easy, all I need is to add more machines.
The logic behind it was also being able to use spot-instances on Amazon, which are much cheaper.
So, as always, I thought I would document the process and open source it.
Challenges
What are the challenges we are looking at, when we want to do something like that.
Awareness of RVM, Ruby version
Awareness of Bundler and Gem versions.
Those are things we need to consider.
Breaking down the process into pseudo code.
So, let’s break down the process into pseudo code
Go to the temp folder
git pull from origin + branch you want
Copy all files from temp folder to current folder
Go To current folder, bundle install
Start God
RVM Wrappers
For the last 2 parts of the process, if you try to do just bundle install --deployment for example, the script will fail, it does not know what bundle is, it resolves to the default Ruby installed on the machine at this point.
Since want the last 2 processes to go through RVM, we need to create a wrapper script.
#!/usr/bin/env bash
if [[ -s "/usr/local/rvm/environments/ruby-1.9.2-p290" ]]
then
source "/usr/local/rvm/environments/ruby-1.9.2-p290"
exec god "$@"
else
echo "ERROR: Missing RVM environment file: '/usr/local/rvm/environments/ruby-1.9.2-p290'" >&2
exit 1
fi
I did the same for bundle which generated this file: (called bootup_bundle)
#!/usr/bin/env bash
if [[ -s "/usr/local/rvm/environments/ruby-1.9.2-p290" ]]
then
source "/usr/local/rvm/environments/ruby-1.9.2-p290"
exec bundle "$@"
else
echo "ERROR: Missing RVM environment file: '/usr/local/rvm/environments/ruby-1.9.2-p290'" >&2
exit 1
fi
The files are created in vim /usr/local/rvm/bin/.
Now that we have the wrappers, we made sure the boot up process will be aware of RVM and we can continue.
RC files
If you want something to run when the machine starts you need to get to know the rc files.
Usually, those files are located in /etc/ folder (may vary of course)
The files are loaded and executed in a specific order
rc
rc{numer}
rc.local
If you want something to run when ALL the rest of the machine bootup process ran, you put it at the end of the rc.local file.
One gotcha here is that the file must end with exit 0, or your script will not run at all, and good luck finding out why :P
I put this line at the end, right before the exit 0
sh /root/start_workers.sh
Finalize
Now that I have the wrappers, I have the line in rc.local I can finalize the process with creating the shell script.
This is what the script looks for me
cd {temp_folder}
git pull origin {branch_name}
cd ..
cp -apRv {source_folder}/* {destination_folder} --reply=yes
echo "Running bundle install"
cd /root/c && /usr/local/rvm/bin/bootup_bundle install --path /mnt/data-store/html/gogobot/shared/bundle --deployment --without development test
echo "Starting RVM"
/usr/local/rvm/bin/bootup_god -c /root/c/god/resque_extra_workers.god
The god file to start the workers is standard.
rails_env = ENV['RAILS_ENV'] || "production"
rails_root = ENV['RAILS_ROOT'] || "/mnt/data-store/html/gogobot/current"
WORKER_TIMEOUT = 60 * 10 # 10 minutes
# Stale workers
Thread.new do
loop do
begin
`ps -e -o pid,command | grep [r]esque`.split("\n").each do |line|
parts = line.split(' ')
next if parts[-2] != "at"
started = parts[-1].to_i
elapsed = Time.now - Time.at(started)
if elapsed >= WORKER_TIMEOUT
::Process.kill('USR1', parts[0].to_i)
end
end
rescue
# don't die because of stupid exceptions
nil
end
sleep 30
end
end
queue_name = "graph_checkins_import"
num_workers = 10
# FB wall posts
num_workers.times do |num|
God.watch do |w|
w.dir = "#{rails_root}"
w.name = "resque-#{num}-#{queue_name}"
w.group = "resque"
w.interval = 2.minutes
w.env = {"QUEUE"=>"#{queue_name}", "RAILS_ENV"=>rails_env, "PIDFILE" => "#{rails_root}/tmp/resque_#{queue_name}_#{w}.pid"}
w.pid_file = "#{rails_root}/tmp/resque_#{queue_name}_#{w}.pid"
w.start = "cd #{rails_root}/ && bundle exec rake environment resque:work QUEUE=#{queue_name} RAILS_ENV=#{rails_env}"
w.log = "#{rails_root}/log/resque_god.log"
w.uid = 'root'
w.gid = 'root'
# restart if memory gets too high
w.transition(:up, :restart) do |on|
on.condition(:memory_usage) do |c|
c.above = 350.megabytes
c.times = 2
end
end
# determine the state on startup
w.transition(:init, { true => :up, false => :start }) do |on|
on.condition(:process_running) do |c|
c.running = true
end
end
# determine when process has finished starting
w.transition([:start, :restart], :up) do |on|
on.condition(:process_running) do |c|
c.running = true
c.interval = 5.seconds
end
# failsafe
on.condition(:tries) do |c|
c.times = 5
c.transition = :start
c.interval = 5.seconds
end
end
# start if process is not running
w.transition(:up, :start) do |on|
on.condition(:process_running) do |c|
c.running = false
end
end
end
end
Profit
Now, it’s a completely automatic process, all you need is to run as many machines as you need, once the machine is booted up, it will deploy code to itself, copy files, install gems and run workers.
If you want to restart the process, add machines or anything else, you can do it with the click of a button in the AWS UI.
All in all, the configuration is awesome and I completely love it, it’s the longest time period I have ever used Vim. Considering that the second place is about 5 minutes, that’s super impressive.
I only have one problem with it.
I took the spec running functions from Gary Bernhardtdotfiles, I just remapped the keys differently.
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
" RUNNING TESTS
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
map <leader>' :call RunTestFile()<cr>
map <leader>; :call RunNearestTest()<cr>
function! RunTestFile(...)
if a:0
let command_suffix = a:1
else
let command_suffix = ""
endif
" Run the tests for the previously-marked file.
let in_test_file = match(expand("%"), '\(.feature\|_spec.rb\)$') != -1
if in_test_file
call SetTestFile()
elseif !exists("t:grb_test_file")
return
end
call RunTests(t:grb_test_file . command_suffix)
endfunction
function! RunNearestTest()
let spec_line_number = line('.')
call RunTestFile(":" . spec_line_number . " -b")
endfunction
function! SetTestFile()
" Set the spec file that tests will be run for.
let t:grb_test_file=@%
endfunction
function! RunTests(filename)
" Write the file and run tests for the given filename
:w
:silent !echo;echo;echo;echo;echo;echo;echo;echo;echo;echo
:silent !echo;echo;echo;echo;echo;echo;echo;echo;echo;echo
:silent !echo;echo;echo;echo;echo;echo;echo;echo;echo;echo
:silent !echo;echo;echo;echo;echo;echo;echo;echo;echo;echo
:silent !echo;echo;echo;echo;echo;echo;echo;echo;echo;echo
:silent !echo;echo;echo;echo;echo;echo;echo;echo;echo;echo
if match(a:filename, '\.feature$') != -1
exec ":!script/features " . a:filename
else
if filereadable("script/test")
exec ":!script/test " . a:filename
elseif filereadable("Gemfile")
exec ":!bundle exec rspec --color " . a:filename
else
exec ":!rspec --color " . a:filename
end
end
endfunction
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
" RUNNING TESTS
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
As you can see, I mapped ,; to run line specs, and ,' to run the entire spec file.
The problem is, that the terminal exists immediately after the result, does not wait for me to hit Enter or anything.
You can see the problem demo in this YouTube Video
As a developer, there are things that never mind how many times you’ll do them, you will never ever remember how to do it exactly without googling for a couple of minutes or looking through the last executed list of commands on the terminal.
For me, one of those things is how to copy all files and folders from source to destination with automatic reply yes to overwrite requests.
I am working with rake tasks a lot, I love the simplicity of creating one and just running one on the server.
For the real heavy lifting I am using a queue system of course, but when I just want to throw something in the queue for example, I will usually create a rake task for it and run it on one of the servers.
I use quick and dirty puts messages to log the progress.
For example I could have a rake task like this:
User.find_each do |user|
puts "Going over user: #{user.id}"
Resque.enqueue(...)
end
To run this on the server, I just ssh into is and then I do screen so when I log out of the server the session will save the process and not kill it.
I have been working with ZHell pretty much since the first day that I have been using a Mac.
The use I make of to grew with time and definitely the biggest switch was that I moved to using dotfiles for settings in a single location.
I forked Zach Holman dot files here, and I have been adjusting and customizing it ever since.
I am using Ruby/Rails for my everyday work, both for consulting and open source projects, the one thing that I see people do and I don’t like is committing configuration files with password and sensitive information into source control.
When I can, I try to avoid it, so I developed a very easy way to manage my per-project environment variable without going through a huge file.
All of my zsh files are located in a single folder in my dotfiles.
As you can see, there’s a special folder there called projects, in which I put a lll of my project specific setting like tokens, passwords and other things.
For example, here’s what a project file might look like:
One thing that is very easy to forget, is that if you open source your dot files (and you should) don’t forget to ignore those files and don’t commit them.