Exploration Through Example

Example-driven development, Agile testing, context-driven testing, Agile programming, Ruby, and other things of interest to Brian Marick

191.8

⇒

167.2

⇒

186.2

183.6

184.0

183.2

184.6

Sun, 22 Aug 2004

Books I mentioned

In my XP/Agile Universe keynote, I cited a bunch of books. Someone asked that I post them, so here they are.

One bit of evidence that test-driven design has crossed the chasm and is now an early-mainstream technique is JUnit Recipes, by J.B. Rainsberger. This is a second-generation book, one where he doesn't feel the need to sell test-driven design. He's content (mostly) to assume it.

I asked how it could be that Agile projects proceed "depth-first", story-by-relatively-disconnected-story, and still achieve what seems to be a unified, coherent whole from the perspective of both the users and programmers. My answer was the use of a whole slew of techniques.

At the very small, programmers are guided by simple rules like Don't Repeat Yourself. Ron Jeffries' Adventures in C# is a good example of following such rules. Martin Fowler's Refactoring is the reference work for ways to follow those rules.

At the somewhat higher level, we find certain common program structures: patterns, as described in Design Patterns. Joshua Kerievsky's new Refactoring to Patterns shows how patterns can be the targets of refactoring. You can think of patterns as "attractors" of code changes.

There are even larger-scale structures as described in Eric Evans's Domain-Driven Design, Fowler's Patterns of Enterprise Application Architecture, and Hohpe&Woolf's Enterprise Integration Patterns.

I was resolutely noncommittal about whether the larger-scale patterns can emerge from lower-level refactorings or whether pre-planning is needed. I tossed in Hunt and Thomas's The Pragmatic Programmer both because it (and they) speak to this issue and because it seems to fit somehow in the "architectural" space.

Finally, I addressed an issue that's been weighing on my mind for more than a year. Dick Gabriel has said (somewhere that I can't track down precisely) that there are two approaches to quality. The one approach is that you achieve quality by pointing out weaknesses and having them removed. The other is that you build up all parts of the work, including especially (because it's so easy to overlook) strengthening the strengths.

Testers too often take the first approach as a given. They issue bug reports, become knowledgeable discussants about the weaknesses of the product, and think they've done their job. And they're right, on a conventional project. I've decided I don't want that to be their job on an Agile project; instead, I want them to follow the second approach. I want them to apply critical thinking skills to the defense and nurturing of a growing product.

But what could that mean? How can it be done? I proposed mining the knowledge of other skilled laborers for ideas. Latour and Woolgar's Laboratory Life: the Construction of Scientific Facts describes how teams of scientists collaborate to produce their primary work product: published and widely-cited scientific papers. (I've written about that earlier.)

I mentioned another source: writers' workshops as used in the creative writing community and imported into the patterns community. Richard P. Gabriel's Writers' Workshops and the Work of Making Things is the reference work here. (See also James Coplien's online "A Pattern Language for Writers' Workshops".

## Posted at 09:13 in category /2004conferences [permalink] [top]

The purpose of a keynote

Back when I didn't give keynotes, only attended them, I thought they had two purposes. The first was to let me get near some godlike figure - your Herb Simon, your Guy Steele - and have wisdom flow from their head into mine. The second was to be inspiring.

I now realize that neither of those is to the point, though the second is closely tied up with the real purpose of the keynote, which is to provide a theme for the conference that threads through all the conversations that make it up. People should refer to the keynote in hallway talk. Other presenters should refer to the keynote's theme as they speak their piece. Questioners should ask presenters questions informed by the keynote. Collectively, the theme gets amplified and people resolve to do things about it.

## Posted at 09:13 in category /2004conferences [permalink] [top]

XP/Agile Universe 2004

I had a very good conference last week, and it seemed that everyone I talked with did too. Thinking back on it, I'm reminded of a book by Randall Collins, The Sociology of Philosophies: a Global Theory of Intellectual Change. As a teensy part of that book, he talks about how important coming together is for intellectuals and members of schools. It's from that coming together, that personal contact, that people get the emotional and creative energy to push the school forward. I'm fired up.

The local folk in Calgary did a fantastic job. They set a friendly and welcoming tone. Whatever glitches there may have been were hidden from the participants. Plus they gave me a cowboy hat.

As was announced at the Agile Development Conference, it and XP/Agile Universe will be merging next year. The united conference will be named Agile United, and it will be held in Denver.

## Posted at 07:30 in category /2004conferences [permalink] [top]

Mon, 09 Aug 2004

Workshop reminder: Tests as Documentation

I've posted before about a workshop that Jonathan Kohl and I are doing at XP/Agile Universe titled "Tests as Documentation". If you were planning to come, let me remind you of this: "We encourage anyone working on a software project who writes tests to [...] bring their tests and code with them."

It'd be a good time to start picking some tests to bring. Thanks.

## Posted at 14:54 in category /2004conferences [permalink] [top]

Sun, 08 Aug 2004

Workshop reminder: Who Should Write Acceptance Tests?

I've posted before about a workshop that David Hussman, Rick Mugridge, Christian Sepulveda, and I are doing at XP/Agile Universe titled "Who should write acceptance tests?" If you were planning to come, let me remind you of this, from the blurb:

Participants will be encouraged to have prepared positions or accounts of experiences regarding the topic. Each participant will be invited to give a brief statement regarding her opinions, experiences or questions related to the workshop topic. The workshop organizers will then facilitate discussions, based on topics derived from the opening statements.

We're not requiring advance submissions or even written positions, but the workshop will be better if people think of their brief statement in advance. Thanks.

## Posted at 16:27 in category /2004conferences [permalink] [top]

Fri, 12 Mar 2004

Exploratory testing and empathy

Exploratory testing is the time where team members get to put on their customer persona and try to use what they have created. I don't know of a better way to develop empathy for the end users.

-- Jeffrey Fredrick

## Posted at 15:11 in category /2004conferences [permalink] [top]

Thu, 11 Mar 2004

The phrase 'exploratory testing'

I don't like the word "test" in "unit test" or "customer test". It leads to endless repetition of sentences like "But don't forget that unit tests aren't really about testing, they're about design." The listener is perfectly justified in asking, "Then why not pick a name that doesn't mislead?" To which, the only real answer is historical accident.

I haven't been enormously comfortable with my use of the phrase "exploratory testing" either. Exploratory testing has historically been a fast and flexible and creative way of finding bugs that matter. My goal for it (in Agile projects) is different. It's to expand the range of possible future stories from which the business expert will choose. Some of those stories might be bug fixes like "make it such that the operating system doesn't crash when you lay a manual on the keyboard and the keys repeat more than about 100 times"¹. But I'm much more interested in stories that are feature requests, ones that suggest adding positive business value rather than removing negative business value.

For a time, I was calling that "exploratory learning" to emphasize that it will use some of the techniques of exploratory testing while still being a different sort of thing. But the name didn't catch on, so I reverted to "exploratory testing." But I'm still unhappy about it.

In the Agile Testing mailing list, Randy MacDonald wrote:

Don't build a system where exploratory testing is possible. Do build a system where exploratory design is possible. In fact, build the system via exploratory design.

I'm thinking that "exploratory design", perhaps with another adjective tacked on front, is more what I'm looking for. So, if I start using that, this will remind me to give Randy credit for the term.

¹ Not apocryphal. I actually crashed Gould's real-time OS that way sometime in the 80's.

## Posted at 07:53 in category /2004conferences [permalink] [top]

Fri, 05 Mar 2004

XP/AU events

There'll be a lot going on, testing-wise, at this year's XP/Agile Universe. I'll be helping out with three events on the program.

• Jonathan Kohl and I will be doing a workshop on tests as documentation. We want to look at examples and talk about how tests can be better documentation.

• Bret Pettichord, Paul Rogers, Jonathan, and I will be doing a tutorial called "Approaches for Web Testing". It will revolve around the open source WTR framework, which is a set of Ruby classes that drive IE through COM. One of the things I find most interesting about WTR is the way it lends itself to exploration through the Ruby interpreter.

  This will be a hands-on tutorial. No starting knowledge of Ruby required, and we expect that what you learn can be applied to other, lesser, languages.

• Finally, I'll be giving one of the keynotes. A keynote should be provocative, both in the normal sense and in the sense of "provoking conversation". I like keynotes that people keep referring back to all through the conference. That's the kind of keynote I'll try to come up with.

  But I also like talks that give people something they can apply the next week when they return to their job. I want to do that too.

Watch this space for more info.

## Posted at 12:54 in category /2004conferences [permalink] [top]

Tue, 24 Feb 2004

Our slant on exploratory testing

For our tutorial on exploratory testing at Agile Development Conference, Elisabeth Hendrickson and I have to answer this question: "What's exploratory testing for, anyway? In an agile project, I mean."

The answer we'll use in the tutorial will be something like this:

We advocate Exploratory Testing as an end-of-iteration ritual in which the Customer, programmers, and other interested stakeholders have the opportunity to take the new Business Value for a test drive to discover New Information.

What we'll want to do is simulate that experience in the tutorial. Our first thought was to do it on (surprise!) software. But there are two problems. First, getting everyone's machine working, getting the software running on it, dealing with configuration problems, etc. - that all takes a good half an hour. That is, maybe 15% of our available time. Ouch.

Second, we can't show the results of the exploratory testing. We can't show a Customer saying, "Oh! What a good idea! Let's put that in right now" - and then having it put in, and then having the Customer see it in action. We can't go through a full iteration, much less more than one. Yet, if exploratory testing's role is largely - as I believe - about shortening the project's feedback loop, we'd be doing a bad thing if we didn't close the loop.

So what we're planning on doing is to have teams of people design a game. The game will be one that demonstrates some property of Agile development. When we tried this out ourselves, we wanted to devise a game that puts across how and why test-driven design feels slower, starts out slower, but catches up in the end.

After the game is designed, groups will divide. Two people will stay, the others will go to join other groups. The changed groups will then playtest the game. They'll use a couple of exploratory techniques we'll describe. They'll come up with new information, things like:

• Wow, so that's what that feels like.

• Hmm... I've changed my mind.

• Oh, this is missing something.

• Gee, that was unexpected.

• That didn't work.

• Maybe what we deferred was more important than we thought.

Then that group will stay together and add something new to the game. Then there'll be another round of playtesting, with a couple of new exploratory techniques.

We're aiming for three things. First, a "Wow, so that's what that feels like" reaction applied to exploratory testing itself. Second, a desire to try it out at a home company. Third, to provide some concrete techniques that apply to software.

And, if we're lucky, we'll get some nice games that really make points well.

Our plans may change.

## Posted at 21:14 in category /2004conferences [permalink] [top]

Mon, 16 Feb 2004

Exploratory testing at Agile Development Conference

Elisabeth Hendrickson and I will be giving a tutorial titled "Exploratory Testing for Agile Projects" at Agile Development Conference, which will be held June 23-26 in Salt Lake City, USA. We'll be using the time until then to bash together my ideas about how exploratory testing will mutate in the specific context of agile projects, Elisabeth's ideas about the same, and Elisabeth's vastly greater experience with exploratory testing. I'll be using this blog to talk about some of what we're doing, partly to help me figure things out, partly to help me learn how to articulate ideas. (I worked with Elisabeth on the tutorial last Saturday, and I was remarkably inarticulate.)

## Posted at 07:36 in category /2004conferences [permalink] [top]