Exploration Through Example

Someone asked me for advice on how to get invited to speak at conferences. Key advice:

• become a valuable participant in a niche field,
• grow along with the niche,
• become a good speaker.

I’ll cover the first two in this note. The details are somewhat particular to my personality and the lucky breaks that came my way. But some of them would work for other people.

The context-driven testing school makes a good example of finding a growing niche. I was an early participant in the USENET group swtest-discuss. On it, I made some naive statements about test automation. Via email, Cem Kaner convinced me that my experience was narrow, that my advice was well-tailored to that narrow environment, and that I was a doofus for thinking it would work for all environments. And there you have it: early experience with what came to be called the context-driven school, and the attention of the person who was to become the leading light. By becoming active with Cem and the people who gravitated to him, I had the opportunity to succeed along with the movement. If I’d tried to make an equivalent impact in the broader world of software development, I’d have failed: too much competition.

Just being there isn’t enough, though. I was involved in software patterns early on, because I was a graduate student of Ralph Johnson’s and participated in his reading group, where we read parts of A Timeless Way of Building, A Pattern Language, and early drafts of Design Patterns. So I could have been a figure in the patterns movement, but I was not. Why not? I was too passive; I did not capture and propagate the zeitgeist.

Contrast patterns with the context-driven school. An early hit of mine was the paper “Classic Testing Mistakes“. Nothing in that paper is original to me. All the mistakes were things people—especially people clustered around Kaner—were saying. What I did was (1) write it down, and (2) make it memorable. I did the latter by giving it a catchy title (and I even lifted the idea for the title from Steve McConnell, I think).

The same is true of “Marick’s test matrix“. At the time (2003), some people were already making a big deal out of the fact that test-driven development was a design technique, that it was at least as much about helping programmers as finding bugs. So the distinction betwee programmer support and product critique was out there among the sort of people who chatter on mailing lists or in small groups at conferences. The distinction between business-facing and technology-facing tests was even more common, due to XP with its customer tests and unit tests.

My contribution? In 2003, those of us who thought of ourselves as context-driven testers and also Agile enthusiasts were fighting a battle we’d already been fighting in the conventional testing world: making exploratory testing respectable. Also Cem Kaner was at the conference pointing out that the conversation about testing in Agile was short-changing “ility” or “nonfunctional” or “para-functional” testing: things like usability, performance, scalability, security, etc. We were talking about all these issues in some little group, Ward Cunningham said something that made me think of crossing the two axes, which—voila—seemed to leave room for two more types of testing, especially once I fiddled around with the names of the two dimensions. (I think the names are almost as important as the visual effect of the axes, especially the distinction between “support” and “critique”.)

So, again, my contribution was in the presentation of ideas that were already floating around in various people’s brains. Call me a popularizer—a Malcolm Gladwell-ish figure. That’s not to say that I didn’t have ideas of my own—though it doesn’t feel right to take sole credit for anything—but that being a popularizer has been important. Popularizers get invited to speak.

So do idea generators. It seems to me that I’m not so much someone who’s come up with any great new ideas as someone who’s good at looking at the familiar from an odd perspective. I get that odd perspective from reading odd things, such as sociology of science or literary theory, and asking “Suppose this point of view were true: what would it mean for building software?”

For example, people like Feyerabend (Against Method), Lakatos (Methodology of Scientific Research Programmes, and Latour (Laboratory Life) point out that science does not work the way we were all taught it should (scientist proposes theory, experimentalist tries to disprove it, if experiment disproves it scientist starts again). So if we think of testing as mimicking science (which many do), what would happen if we mimicked the (claimed) reality instead of the theory?

Well, for one thing we’d learn that much critiquing of a paper happens before publication, by the authors’ physically-present colleagues, who are trying to anticipate likely attacks and suggest ways to protect the paper against them. Carrying that over to software, there are some lessons that seem obvious (now, but were less obvious seven years ago): testers in the team room, testers helping to define the product by asking the right questions before the coding happens, testers looking at multiple “drafts” of the product from a new perspective each time (which I would claim argues in favor of exploratory testing).

If you want to keep going with the analogy, you start to ask more questions about specialists and generalists. I don’t think there are any scientists whose only job is to critique papers. Every paper-critic is also a paper-writer. Now, there are certainly scientists who are exceptionally good at critique (Hello, Dr. Constable!), or even at particular kinds of critique (like of the statistics), but they are always peers and always capable of pitching in. (For example, the stats-master might be able to help with the statistics, or someone might volunteer to run a particular assay that would protect the paper against a particular criticism.) And so on.

Whenever possible, ground the ideas in the concrete. It’s not always possible, but it’s sure nice if you can show a working example or tell a story of your experience with the idea. Not only will this save you from spouting some stupid ideas, people are much more receptive to ideas that come packaged with “things”.

If you don’t have an example of the idea in action, you can still view a past experience through the lens of the idea. Now that you’re thinking in way X, how do you reinterpret what happened? What changes in the way you did things then does idea X suggest now?

One thing I’ve done that’s been key is to help others write. I think I wouldn’t have been invited to the workshop that wrote the Agile Manifesto had not I not workshopped Martin Fowler’s books in Ralph Johnson’s reading group. By helping someone with a book, article, or paper, you’ll not only earn some gratitude, you’ll learn a great deal about how to write and what styles of getting an idea across work for you.

There are probably more people who can create good ideas than who can express them, so specializing in expression isn’t a bad way to stand out from the crowd.