I’m the type of technologist that messes around with a lot of different technologies, some for fun, some to help out friends, and sometime for profit.

Recently, I came across a mix of all 3.

Here is the context:

  • Private Github repository
  • Rails application
  • Jenkins/Hudson CI ( from here on out, i’ll call it Jenkins, as I’m more for Jenkins then hudson )
  • Test and display code coverage results

Now I know some of these things pretty well ( the reason why I was asked to consult ), and some not so much.  I know github pretty well, but don’t have any private repositories setup, so that means dealing with authorization of a project.

I’ve dabbled with rails, but wouldn’t call myself an expert by any means, and automated testing of rails projects even less.  I do know a significant amount about code coverage from other languages I’ve work on before, so that part wasn’t very new.

I do know Jenkins/Hudson quite well, as I was an early adopter close to 4 years ago.

First the setup.  I won’t talk about setting up Jenkins/Hudson, since there are a thousand and one posts/tutorials out on the web already.  Instead I will try to focus on the specifics.

To access a private repo, you need to have ssh keys properly setup ( as all the github tutorials will explain ) for the user that JENKINS is running as, so that means creating a new ssh key for the user running JENKINS ( not your github user ) and uploading this key to your github account.

When configuring your jenkins job, select git as the source control management, and enter the url to your github repository ( i.e. git@github.com:<username>/<project> ).  If your keys are properly setup, the clone should work without a problem.

I didn’t end up using the Rake plugin for Jenkins, as i wanted to run a “bundle install” before running the tests ( in case a new gem was added to the list ), so i used shell commands instead to build ( i.e. export RAILS_ENV=test; bundle install; rake test ).  The one plugin you will need is “Hudson ruby metrics plugin“, to actually display the graphs of your code coverage.

Getting the code coverage information.  The code coverage people mostly talk about on the web is called rcov.  It looks good, and works for 1.8, but the project in question is 1.9, oops.  there is a project out there called simplecov, that seems to work really well, but the output format isn’t what jenkins expects.  Great, another headache.  In steps simplecov-rcov, its just a formatter for simplecov, that generate rcov style reports.  So lets add this to our test/test_helper.rb

require 'simplecov'
require 'simplecov-rcov'
SimpleCov.formatter = SimpleCov::Formatter::RcovFormatter
SimpleCov.start 'rails'

The reports now generate properly.  Lets throw this all to jenkins.  I was not the one to initially setup the Jenkins installation, or the job, so i tried to not change anything unnecessary( including the name “test” ).  Things seem to run, the tests go by, but the coverage files are not been generated properly……enter a couple hours of head scratching/puzzlement.  check permissions, run with trace, try everything.  Somebody then mentioned, maybe the name “test” is the problem, as rails uses convention over configuration, and test is a “reserved” word.  I rename it to something more meaningful, and voila,  things work marvelously.

I don’t know if the problem originates from ruby, rails, or the plugin.  But because the path had test twice ( once because the name of the jenkins job, and once for the actual test folder for the rails tests ), it couldn’t perform the introspection into the code to calculate the coverage.  Lesson learned, convention over configuration works well, until it starts deducing too much from parent paths.