Setup vs. Action for Ruby tests

A WORK IN STAGNANT PROGRESS

I’ve been thinking about how to express behaviour in tests in a natural and fluid way. One problem I’ve noticed is that with the classes xUnit setup -> test -> teardown structure is that the setup often ends up doing more work than it perhaps should.

This forms some of the background behind my work on Kintama.

The should_change straw

I think a good example of this is the should_change macro that shoulda used to provide. I’ll quote from their blog post:

Consider this example of should_change:

context "updating a post" do
  setup do
    @post = Post.create(:title => "old")
    put :update, :post => {:title => "new"}, :id => @post.to_param
  end

  should_change("the post title", :from => "old", :to => "new") { @post.title }
end

This reads well and seems to be fairly straightforward. Sadly, this doesn’t work because of how should_change works internally (hence the confusing part). It would actually need to be written like this:

context "given a post" do
  setup do
    @post = Post.create(:title => "old")
  end

  context "updating" do
    setup do
      put :update, :post => {:title => "new"}, :id => @post.to_param
    end

    should_change("the post title", :from => "old", :to => "new") { @post.title }
  end
end

The @post instance variable needs to be assigned a context above the context containing should_change. I told you it was confusing.

Basically, because should_change needs to know the state before the setup ran, you need to nest another content. It’s not the worst thing in the work, but blurgh.

Anyway, the shoulda authors declared this a smell and deprecated the method, while I disagreed because while I agreed that the mechanism was crufty, some of the conclusions they drew about the invalidity of testing change were a bit tenuous.

Structural flexibility

Perhaps, I wondered, there was a better way of structuring the tests that would make it more natural to write tests including things like should_change? Surely nesting isn’t the only bit of sugar we can sprinkle on “setup -> test -> teardown”?

The problem was that the tests above were conflating the “setup” with the “action” that was being tested. Perhaps they are different? The first idea that popped into my head was sketched like this:

[snip 'codeshould_change' cannot be found]

That seems to avoid the “ugly” nested context, and is simultaneously clearer about what we’re actually interested in testing here.

So, combine this idea with a general frustration at the way that test-unit works¹ and a continued unwillingness to drink the rspec koolaid², I did what it turns out a million other people have done, and started exploring my own test framework, Kintama.

Using action in controller tests

But… some objections to should_change have nothing to do with the clunkiness of defining nested contexts. They, quite fairly, point out that it encourage (or is symptomatic of) state-based testing; poking into the database to see whether some value has changed. And often that is a code smell.

Perhaps the should_change aspect is a red herring.

What was really driving the exploration of action was avoiding repetitive tests where some parameters of some action were changing:

[snip 'codecontroller_test' cannot be found]

The action is called after all the (nested) setups for a test have run, but just before the test itself. We can log a user in, and set the attributes used in the action, or we can not log anyone in, and in each case write regular tests to describe how the object should behave.

I think this probably requires a bit of internalisation, but I like the clarity of declaring “right, this is the specific thing that is under test right now, and here is how it should behave in various different situations”.

Next steps

I want to explore using action in tests more; this almost certainly means using it in anger on an actual software project.