I’ve been aware of Ward Cunningham’s “Fit” framework (
http://fit.c2.com/) for some time. I’ve never actually used it, because it seemed “hard” and my business analysts had their own way of doing things.
My current project introduced me to “FitNesse” (
http://fitnesse.org/), a wiki which integrates the Fit framework with wiki pages, and forced me to learn to use it for real enterprise-scale projects, both interactively and as an automated test suite to be run as part of our continuous build process.
Boy, have I been missing out; this thing rocks!
At the core, FitNesse is a wiki. It’s important to remember that, because as it’s presented in the documentation, it’s primarly a test tool. But here’s the thing: if you just type your requirements into the wiki, it more-or-less demands that you write tests, too. That is, that you express your requirements as something that the developers must test. That has two huge benefits: 1) it makes sure developers speak the same language as the requirements, and 2) it forces requirements to be testable. Because it’s a wiki, as developers and analysts come to a shared understanding of the requirements, either can easily tweak the pages to say what needs saying, and the tests to express those requirements succinctly.
As requirements and associated tests are developed in FitNesse pages, you can check them in to source control, run them regularly, and there’s no need to back-trace test to code to requirements–if the FitNesse tests run, you’ve met all the requirements they embody. It’s about the shortest distance between requirements and a quality deliverable I have ever seen, and one of the lowest overhead. Rather than checking our FitNesse pages in “at the end of an edit”, we have a scheduled job which checks in any changes at the end of every day.
The examples on the Fit and FitNesse web sites are simple; I encourage you to go there now and have a look. These are great for learning, but they don’t express the real power of the tool.
Here’s an example from my current project in all its enterprise-messy glory. This is what real enterprise application testing is about. The image below is about 2/3 of the testing page for one scenario (click the image for a full-size rendering. It’s large (about 100k, and about 3x my screen)). Most of the page sets up the test conditions. Along the way, there are tests to verify the (very complex) input data are correct; “Read Tax Rate Area”, for example, generates numbers which aren’t in any specific location–they’re summed over dozens of inputs already in the database. Any heading with a question mark is an invitation to invoke the application code to see what the results are.
Binding this page to the application code requires a “fixture”, which each developer writes. Fixtures are typically quite small. The set of fixtures which executes the tests on the page shown total 596 lines of code; it took me about a day to write, including figuring out exactly how to perform the required setup. I had to write a fixture base class which took another day, but now that I have that written, writing new complex tests will be as simple as writing this one was. This scenario, and 11 others just like it, run using the same fixture and exercises the same code as runs in production. Those 11 scenarios represent most of the edge cases in one kind of billing. There are many more; a total of around 400 scenarios and thousands of individual tests.
To run the test, you simply punch the test button in the upper right-hand corner of the page. I’ll walk through some of the results. First, the top of the page notes that 2 tests succeeded, 2 failed, 2 were ignored completely (usually as a result of failure), and 2 exceptions were thrown. Each row typically represents a test, and that’s not anything like the full population of the page, but exceptions tend to end things abruptly. Secondly, note the “Read Tax Rate Area” block, where two tests ran, and both failed. Thus, the assumptions which are built into results further down the page are proven wrong–the testers need to know this so they can revise their expected answers. One important thing to note is that the framework handled the display; all the fixture code does is respond to “getLipTaxRate()” and “getLiTaxRate()”–names derived from the associated headings–with a text string. The framework colored the cells and placed the “expected” vs. “actual” answers on the page.
Scrolling down, there’s an exception. Yes–I made a mistake in the fixture which, under the right conditions, throws an NPE. I’ll fix that in the next release. On down is a successful test.
That’s what it looks like. The fixture code is easy to write, the requirements are expressed in a way the business people understand, and there’s a man-day or less (usually far less) between describing the behavior in a FitNesse test and writing the fixture which connects the test to the functional code.
In our environment, FitNesse tests are also run automatically as part of our automated build cycle. The tests span hundreds of wiki pages. The results are consolidated into a nice punchy list with counts of successes and failures, much like the top of the first “post-run” page above. For failures, more detail is provided. The XML parsing is done by fixures already available in CruiseControl. I found excellent help for configuring our ant build (and thus our CruiseControl build) at
http://www.jroller.com/njain/entry/integrating_fitnesse_with_cruisecontrol. Thanks, Naresh!
What more could you want in a requirements/code/test/document system? Go forth and experiment. You’ll be glad you did.
