The Official Citrusbyte Blog

With Cucumber, you write stories and user acceptance tests in plain English, something a lot of people seem to like.

Feature: Search
  As a user
  I want to find information
  So I can learn more

  Scenario: Find what I'm looking for
    Given I am on the Google search page
    When I search for "rspec"
    Then I should see a link to RSpec-1.2.4: Home

Then you need to write your “step” files, where you add blocks with regular expressions that must match your plain English user stories and user acceptance tests. Inside those blocks you add your actual Ruby code. If you are using Cucumber for testing Rails or Sinatra, then your step definitions will look like this:

Given 'I am on the Google search page' do
  visit('http://www.google.com/')
end

When /I search for "(.*)"/ do |query|
  fill_in('q', :with => query)
  click_button 'Google Search'
  selenium.wait_for_page_to_load
end

Then /I should see a link to (.*)/ do |expected_url|
  click_link expected_url
end

Now that you have your plain English features files and your step files, you can finally run your tests. As a result, you will get a nice output of the user stories and user acceptance tests in plain English. But just to clarify that: the output is the same plain English stories you wrote at the beginning. The cost for this is to keep in sync two files for mostly the same thing.

In Citrusbyte we have a different workflow. We agree on the user stories with the client, and we put into instructions the user acceptance tests. We do that using Stories, which is a small add on for Contest. It allows you to declare user stories, scenarios and steps using Webrat. The same example, this time with Stories, would look like this:

class SearchStoriesTest < ActionController::IntegrationTest
  story "As a user I want to find information so I can learn more" do
    scenario "Find what I'm looking for" do

      visit "http://www.google.com/"

      fill_in "q", :with => "Citrusbyte Stories"

      click_button "Google Search"

      assert_contain "citrusbyte's stories"

      click_link "citrusbyte's stories at master - GitHub"

      assert_contain "Stories and User Acceptance Tests"
    end
  end
end

When you run that with the stories runner, you get this output:

As a user I want to find information so I can learn more
  — Find what I'm looking for
      Go to “http://www.google.com”
      Fill in “q” with “Citrusbyte Stories”
      Click “Google Search”
      I should see “citrusbyte's stories”
      Click “citrusbyte's stories at master - GitHub”
      I should see “Stories and User Acceptance Tests”

Now you can redirect that output to a text file and email it to your client, or you can use the stories-pdf runner, send the PDF straight to your client and get a delicious cake in return. Don’t believe me? Check this post.

You will have all your tests in one place, you will be writing Ruby, your clients will be happier than ever and the overhead will be minimal. Grab the gem from github and start impressing your clients today!

{ 8 comments }

At Citrusbyte we’ve launched production websites with the major three Ruby testing frameworks: Test::Unit, RSpec, and Shoulda. In our experience using these libraries, we never felt like we found our ideal framework. We wanted to move the company toward a single testing platform, so we looked at what was out there realized none of the current offerings fit what we needed and ended up creating Contest.

Test::Unit

Test::Unit is old and well tested, comes with the standard library ,and is easy to pick. The downside is that after using Shoulda and RSpec we fell in love with nested contexts - without them the code gets too large and not DRY. It leads to pulling out lots of helpers to reuse in your tests, which in turn takes your context setup away from your tests (a similar problem to what occurs with fixtures). As a test suite grows larger, the helper suite grows in a different place and it leads to hard-to-maintain tests.

RSpec

RSpec adds a magic syntax for no good reason. For example, compare these two assertions:

With RSpec:

@user.should have(4).friends

With the rest, using plain old Ruby:

assert @user.friends.size == 4

Or using a helper assertion:

assert_equal 4, @user.friends.size

While the magic syntax is helpful for getting into the BDD mindset, once you get it you can do BDD with any similar tool.

Another point against RSpec is the fact that its codebase is big and somehow unstable. Even if the developers are fast at patching bugs, the testing framework is a place where you really don’t want to deal with bugs. A larger codebase also means that you need more time to understand its internals in case you need to patch it yourself. Finally, RSpec is the slowest of the lot, and with large test suites the difference can amount to many seconds.

Shoulda

Shoulda provides what we want, but also provides features we don’t want to use. We don’t want macros, before_* statements, Rails or ActiveRecord integration. We want our testing framework to test our code, not code from other libraries.

In the end, Shoulda and RSpec give us roughly the same power (in terms of testing, ignoring that spec/mocks is included in RSpec), and Shoulda provides it with far less overhead (code that might need debugging but is difficult to understand). Shoulda forgoes the `foo.should be_valid?` syntax in order to avoid a couple of code smells. See the Shoulda announcement for more information about these code smells and the differences with RSpec.

We found that we actually really like RSpec and Shoulda because of BDD, but not because of their particular functionality or DSLs.

Our Ideal Framework

Reflecting on all these facts, we agreed that our ideal framework should have nested contexts to suit our BDD needs, be as small as possible, stable, and have no unnecessary magic.

As we looked at our needs for a framework, we realized that while RSpec and Shoulda are powerful, we could just add nested contexts to Test::Unit and that would fulfill our criteria. We did that with less than 100 lines of very simple code and called it “Contest”.

Here’s a quick sample of how it looks like:

require "rubygems"
require "contest"

class Array
  def rotate_left(n = 1)
    n.times { push(shift) }
    self
  end

  def rotate_right(n = 1)
    n.times { unshift(pop) }
    self
  end
end

class TestArray < Test::Unit::TestCase
  context "Array" do
    should "rotate elements to the left when sent rotate_left" do
      assert_equal [2, 3, 4, 5, 1], [1, 2, 3, 4, 5].rotate_left
    end

    should "rotate elements to the right when sent rotate_right" do
      assert_equal [5, 1, 2, 3, 4], [1, 2, 3, 4, 5].rotate_right
    end

    should "rotate elements to the left 2 places when sent rotate_left(2)" do
      assert_equal [3, 4, 5, 1, 2], [1, 2, 3, 4, 5].rotate_left(2)
    end

    should "rotate elements to the right 2 places when sent rotate_right(2)" do
      assert_equal [4, 5, 1, 2, 3], [1, 2, 3, 4, 5].rotate_right(2)
    end
  end
end

Contest is available in our labs and at github. Feel free to use it!

{ 7 comments }