Exploration Through Example

When I was in Middle School (~12 years old, around 1971), we did a murder mystery exercise in class. The teacher passed out slips of papers with clues and then shut up. The children milled around aimlessly for a while, comparing slips. Finally, I got fed up, got everyone’s attention, and said, “OK. Everyone with clues about the murder weapon, go over there. Everyone with clues about the victim, over there.” Then I went from group to group, and we quickly solved the mystery.

I learned three things that day.

1. Sometimes people need external organization to get things done.

2. I can do that organizing.

3. People will get mad at me when I do.

The lesson has affected my consulting. I suspect that my daughter’s homework today—identifying the parts of a knight’s armor—will not affect her future in the slightest way. Social studies no longer appears to involve the study of society. Not surprising, I suppose: too much risk of children drawing conclusions that will get their parents mad at the school, and who needs the hassle?

And the New Math rocked, by the way. It was good that I learned simple set theory so young, and the idea that there could be number bases other than ten was a good lesson in questioning the verities.

A continual debate on the agile-testing mailing list is to what degree testers think differently than programmers and are therefore able to find bugs that programmers won’t. Without ever really resolving that question, the conversation usually moves onto whether the mental differences are innate or learnable.

I myself have no fixed opinion on the matter. That’s probably because, while vast numbers of testers are better than I am, I can imagine myself being like them and thinking like them. The same is true for programmers. In contrast, I simply can’t imagine being the sort of person who really cares who wins the World Cup or whether gay people I don’t know get married. (I’m not saying, “I couldn’t lower myself to their level” or anything stupid like that—I’m saying I can’t imagine what it would be like. It feels like trying to imagining what it is like to be a bat.)

However, I’ve long thought that security testers are a different breed, though I can’t articulate any way that they’re different in kind rather than degree. It’s just that the really good ones are transcendentally awesome at seeing how two disparate facts about a system can be combined and exploited. (A favorite example)

Bruce Schneier has an essay on security testers that I found interesting, though it doesn’t resolve any of my questions. Perhaps that’s because he said something I’ve been thinking for a while:

The designers are so busy making these systems work that they don’t stop to notice how they might fail or be made to fail, and then how those failures might be exploited. Teaching designers a security mindset will go a long way toward making future technological systems more secure.

The first sentence seems to make the second false. When I look back at the bugs I, acting as a programmer, fail to prevent and then fail to catch, an awful lot of the time their root cause wasn’t my knowledge. It’s that I have a compulsive personality and also habitually overcommit. As a result, there’s a lot of pressure to get done. The problem isn’t that I can’t flip into an adequate tester mindset, it’s that I don’t step back and take the time.

So, I suspect the interminable and seemingly irresolvable agile-testing debate should be shelved until we solve a more pressing problem: few teams have the discipline to adopt a sustainable pace, so few teams are even in a position to know if programmers could do as well as dedicated testers.

From Andy Pols: the customer who wouldn’t deploy without a test (via Keith Braithwaite).