Exploration Through Example

Example-driven development, Agile testing, context-driven testing, Agile programming, Ruby, and other things of interest to Brian Marick

191.8

⇒

167.2

⇒

186.2

183.6

184.0

183.2

184.6

Sun, 13 Nov 2005

Throwing tests away

In part of "When should a test be automated?", I look at tests that have broken because the intended behavior of the code changed. My urge is to fix them, but I stop myself and ask a question: if this test didn't exist in the first place, would I bother to write it? If not, I shouldn't bother to fix it. I should just delete it.

Here's an example where that practice would have led me astray.

I've been hacking away at the code in RubyFit that parses HTML. I'm changing it so that it will support Rick Mugridge's FitLibrary. To that end, I created a class, TagSlices, that splits HTML text at tag boundaries. For example, the TagSclices of foo<tag x="y">bar</tag>quux would be foo, <tag x="y">, bar, <tag>, and quux.

I'd looked at the Java implementation before starting. That code operates on a lowercased string for tag-matching, but returns chunks of the original string. In the implementation I started moving toward, maintaining the two strings was inconvenient, so I talked myself into thinking I could downcase the original string at the start and work only with that. Stupid (people sometimes do use capital letters in web pages), but I was backing away from a frustrating implementation closely modeled after the Java one - and thus un-Ruby-like and hard to get right. I was so focused on tags that I thought what was OK for them was OK for everything.

I'd generated TagSlices using a set of tests that did not reveal the bug. After I was done, I reran the unit tests for the old version. (I hadn't used them for development because they "chunked" the problem in a way that didn't fit the path I was taking.)

Here's one of those tests:

It failed on the line marked with an arrow. I thought about that. Was the failure due to a bug? No, I'd decided it was harmless for tags to change case. Did any other assertion fail? No. Was the test completely redundant with other tests? It seemed so. So I should have thrown the test away. But I hesitated. After all, the changed behavior was a side effect, an implementation convenience. It would be just as harmless for tags to keep their case and pass the test. Maybe that wouldn't be as hard as I'd thought when I'd started. I looked at the code and it suddenly flashed on me that lowercasing the whole string wasn't harmless at all.

And, moments later, I realized that Ruby is a scripting language, after all; as such, it lives for regular expressions. Maybe in the Java world, it makes sense to search a lowercased string for "<table". In the Ruby world, it's better to search the original string for /<table/i.

So I created a test that talks specifically about case in tags and non-tag text. I made it pass. The old test passed, too. I could have thrown it away. And yet... what else might it uncover someday? So I kept it.

I shouldn't extrapolate too much from a single example, but it makes me wonder. Seven years ago, when I wrote the paper, I was solidly embedded in the testing culture, a culture of scarcity, one in which:

• Automated tests were expensive to write because they had to go through an interface not designed for testing.

• Programming time to fix that was almost entirely unavailable.

• You were never anywhere close to having as many tests as you thought you needed, so the opportunity cost of fixing an old test was high.

Those assumptions are less true today. Because of that, it makes more sense to change old tests on the off chance you might find bugs or learn something. One of my other testing guidelines is to seek cheap ways to get dumb luck on your side. I'm not smart enough to do without luck. (That's not false modesty: I bet you aren't either.) Fiddling with tests is perhaps now cheap enough to be a luck generator.

(P.S. The bug would have certainly been caught by even the simplest realistic use, so it wouldn't have survived long.)

## Posted at 18:02 in category /testing [permalink] [top]

A rant: filenames

Java is a good thing. Rails is a good thing. But just as Java /perpet[ur]ated/ the horrors of StudlyCaps on those of us who like to read quickly, Rails is /perpet[ur]ating/ underscores in filenames on those of us who like to write quickly.

There are legions of examples of people not acting according to their rational self-interest. Yet another is the prevalence of filenames like webrick_server.rb over webrick-server.rb. Does an underscore take more energy to type than a dash? Yes. Does avoiding a typo require more coordination? Yes. So why this pathology?

• C used to be the programmer's lingua franca. Since '-' in C means subtraction, variable names conventionally contain underscores. Did the trailblazer Unix programmers not realize that filenames don't have to follow the same rules?

• Is it because VMS only allowed underscores in filenames, and VMS is just so totally cool?

Whatever the reason, we must consider the result. Thousands upon thousands of people already suffer from Emacs Pinky. To add injury to injury, those fragile pinkies must suffer additional unnecessary damage striking the shift key. How many people have been forced to switch to vi because those underscores pushed them over the edge to pinky RSI? That's a tragedy no caring person can ignore.

You have a lot to answer for, David HH. I was this close to convincing the world to use dashes, and now there's no hope.

## Posted at 18:02 in category /junk [permalink] [top]