The tenth time you say it, decide that it’s wrong
That title seems like a good motto in general. Here’s a specific instance.
I was on a short consulting trip recently. We talked about Fit. I said the two things that I’ve said to clients at least ten times:
“I recommend using Fit when the product director wants to read the tests or when the data is naturally tabular. If the product director doesn’t care or if the test is a do-this-do-that-check-this type test, I’d rather you used xUnit.”
…
“There are no decent HTML editors. The most tolerable I’ve found is OpenOffice, but they’re all a pain when you want to modify tables.”
There’s a contradiction here. If the product director doesn’t care, why would you write a tabular test in HTML? Because using xUnit means you have to keep columns aligned, and that’s a pain. So use HTML and have the browser line up the columns. But I just said that editing HTML is also a pain. And I never gave much thought to which is the greater pain.
Well, now I have. Here’s a set of tests for business rules similar to the ones the client uses (obfuscated so that the company can’t be identified):
|
I’ve written a pile of code to make those tests pass, doing some fiddling with them along the way. Subjectively, making a small allowance for bias, I’m thinking it’s been about the same pain as editing HTML. If I wasn’t a complete novice at TextMate, it would probably be less. (Surely TextMate must have an equivalent of a Back button so that you can flip between files easily even if you haven’t dragged the tabs to be next to each other.)
I even think this test isn’t horribly unreadable for a non-technical reader once you tell her to ignore the starting and trailing gobbledeegook, underscores, and quotes. (And it wouldn’t be too hard to write a script that did that for her, or even one that generated HTML.)
The output of a failing test is much less appealing to a product director, but it’s more appealing to someone actually working with that output (a programmer or tester): it’s got a stack trace, probably one that’s clickable.
This approach would encourage people to write test fixtures in Ruby, which is considerably faster than doing it in Java or C#. And it makes fixture-writing accessible to the well-read tester. In this particular case, the app communicates with the outside world via XML over the network, FTP files, and a database, so there’s no need to link the fixtures into the code. If it worked through a GUI, some remote procedure call interface would have to be added, but I persist in thinking that’s not too much to ask.
If anyone wants to pursue this approach on real code, let me know. You’ll get a 50% break on my consulting rate. I make the same offer about my prototype graphical tests.
April 19th, 2007 at 1:26 pm
Cool!
I would also think that in a decent editor (like emacs ;)) you could have a mode that would make it far easier to keep the indentation right automatically.
April 19th, 2007 at 1:45 pm
I think M-x picture-mode is supposed to do that, but I always mess things up just as bad.
April 22nd, 2007 at 8:00 pm
Re TextMate’s back, the Go to File window lists recently edited files in reverse chronological order, so Command-t, Return is what you want.
-john.
April 23rd, 2007 at 3:30 pm
Spreadsheets are a lot easier to edit than HTML, and I believe Fit can read Excel - would that help?
April 26th, 2007 at 2:24 pm
I only tried Fit-with-excel once years ago, so I have no reasonable opinion. My unreasonable opinion is that it’s excellent for test input, maybe excellent for test editing, OK for test display, but as weak as HTML for the back end (making the program pass, and keep passing, the test).
April 26th, 2007 at 6:30 pm
[…] The sample application is only a realistic sketch. It pretends to be the server part of a system used by travel agents. For my own convenience, I wrote the application in Ruby, but I’m treating it just as I would were it in Java. The tests are written in Ruby for the reasons I gave earlier. […]
April 29th, 2007 at 6:24 pm
As a programmer, i’ve always hated the lack of flexability in FIT: it seemed hard to use because it was hard to make sophisticated tests that still had a simple and readable “top level.”
I’ve been using Ruby for web testing for ages, and I feel that it’s no harder to read, even for non-techies, once you get over the “it’s not in Excel or Word” psychological barrier.
The more user-oriented tests I’ve done in this mold are much like yours, but with “higher-level” support functions, and oriented toward being more “englishy.” E.g., when a customer required he had google analytics working on certain pages:
def test_google_analytics_tracking_code_exists
%w[
/pc/
/pc/mobile/
/pc/shop/
/pc/products/
/ml/XXX
/send-mail/AAAAAA
].each {
|uri| assert_analytics_code_present(uri)
}
end
Do the footers come out in the right language, and have the right data?
def test_footer_en
assert_get ‘/en/page/top’
assert_body_match ‘日本語‘
assert_body_match ‘top‘
assert_body_match ‘XXX Co. Copyright 2006′
end
def test_footer_ja
assert_get_response ‘/ja/page/top’
assert_body_match ‘English‘
assert_body_match ‘top‘
assert_body_match ‘XXX Co. Copyright 2006′
end
Sometimes I name all the pages, so you can do things such as add parameters, post, and make sure that you went to the “main” page or “failed-login” page. For web sites with databases, there are routines to assert that a user is or isn’t in the db, and so on, so that you can test registration.
All this is dead easy in ruby, and if you hide all the support stuff in files in another directory, you can aim a business guy a the folder full of “high level” tests, associate the extensions with a simple text editor, and he can easily examine the current tests at any time.
He still needs to work witha programmer to change the tests, but for other reasons (particularly deciding what to test and how to test it in the first place), he’s going to have to do that anyway.