Exploration Through Example

Earlier today, I found myself talking to Andrew Shafer and Corey Haines about certification. Certifications provided by people who make good money off them beg for distrust. So do certifications of courses or trainers when the certifiers get revenue from that activity. On balance, I think we’d have been better off without the certifications we have, and I agree with the Agile Alliance’s position on certification. (Disclosure: I was the main author.)

However. Here we are, aren’t we?

To improve the current situation, I propose the following, roughly inspired by Consumer’s Union.

• There should be an Agile Alliance program that applies to all courses, not just those that provide certifications.

• This program will, with the Agile Alliance’s money, send spies to sign up for courses. They will anonymously write up detailed evaluations that will be posted on the Agile Alliance website.

• It will also interview a random sample of course attendees. The ones contacted just after the course will be asked for their evaluation. The ones contacted a year after the course will be asked how valuable, in retrospect, the course was. These results will also be posted on the site.

• These interviews and ratings will, over time, allow the writing of summary articles like the ones Consumer Reports Magazine runs. Expect articles about “What to Expect from a Certified ScrumMaster” or “How to select a TDD course”.

• Those running the program will pay careful attention to maintaining the trust of their readers.

It’s possible I’d participate in this. I have no desire to run it. I don’t care that much.

Michael Bolton has asked me to improve the sentence in the headline. Here is my attempt. It stems from two of my beliefs:

• It’s better to focus on the immediate problem than on convincing people of a universal truth.

• Right action does not follow from thinking correctly. Thinking correctly follows from the habit of acting correctly.

The sentence is an aphorism: terse, stylish, true, and deep. I think its proponents value the aphoristic quality as much as they do the underlying sentiment. When I watch them use it, I believe they’d be happiest if they could generate an epiphany (”a sudden, intuitive perception of or insight into the reality or essential meaning of something”). I’ll try and mimic some of the aphoristic terseness at first, even though I am explicitly not going for an epiphany.

Here goes:

1. The setting: an unproductive conversation needs to be redirected. It’s likely that the conversation is happening because some important people are unhappy. But instead of talking about how to make them happy, we’re talking about whether they are right to be unhappy. We’ve come down to—explicitly or implicitly—competing definitions of quality.

2. There is a brief lull in the conversation. Our hero speaks.

“Different people care differently.” (beat)

Because the sentence doesn’t obviously have anything to do with quality, it has a desirable “huh? where did that come from?” effect that keeps people from jumping into our hero’s pause-for-emphasis.

3. The hero speaks the following sentence with an “opening up / giving” hand movement, the sort where you bring your hand up and out, rotating it from palm down to palm up. This counters the harshness and abruptness of the first sentence: Diff-rent pee-pul judge diff-rent-ly.

“So different people will judge our product differently.” (beat)

A little rhetorical flourish in the parallelism of the two sentences. The “so” at the beginning switches the meter to more iambic, gives it a little forward momentum, opens it up into a more conversational tone. “Judge” is a strong, active word (as is “care”), and focuses attention on what people are doing that the group must react to.

4. Someone tries to jump into the pause. Our hero gains time for one more utterance with a “just one more thing” head-tilt-back, audible intake of breath, and soft palms-out rotation of the hands, forefinger up.

5. The next bit takes long enough to give our hero a chance to make contact, via hand and head movements, with the people who are bored sick of the argument and have been sitting silently while the true believers battle it out. They’re being invited to break their silence to lend support.

“I predict that if we figure out which people we care about, and figure out what they care about, and how we can know their judgments before we get surprised like we were, we can live long and happy lives without ever agreeing what quality is. Let’s give it a try, OK?”

Long—because the problem isn’t simple. A program for action. Inviting a group of arguers to become “we” again.

The whole structure of the three parts is moving from authoritarian (”a fact you must recognize is…”) and abrupt to light (”long and happy lives”) and consensus-seeking (”we”, “OK”, ending on a rising question-tone).

That’s the title of my Agile2010 workshop. Here’s the description:

We’re not shy about saying that testers, programmers, and everyone else in the team needs to become fluent in the business domain. But we’re blasé when programming practices and programming knowledge are the exclusive province of the programmers. That hampers communication and hurts teams. By simply having people stand up, move around, and speak messages to each other, this workshop will show anyone who attends how advanced object-oriented programming and test-driven design works. Although tailored to non-programmers, programmers may find it useful too.

It’s often the case that a class has one or more collaborators that are used by many of its methods. When you want to override one for testing, you can either pass the test double into the constructor (probably as a default argument) or you can smash the double into an instance variable you happen to know points to the collaborator.

I used to use the constructor approach, but I increasingly find myself using the Hulk! Smash! approach. It looks like this…

The object-to-be-tested names its collaborators and gives them the values they’ll use in production:

I do this because I think of the collaborators as a static, declaration-like property of the class. (If I were ambitious, I’d convert collaborators_start_as into a class-level declaration like this:

I haven’t been that ambitious yet, as it turns out.)

Each test declares which collaborators it overrides:

Note that (in Ruby), I can even mock out classes, so I don’t have to go through contortions to pass in instances that an outside caller really would have no particular interest in:

With several of the mocking packages available to me, I wouldn’t even have to go through the indirection of putting the class ThingSource into an instance variable. I could do this:

I’m not bold enough to do that yet.

One of my pet peeves is how people get all obsessed with marginal tax rates when no one actually pays them. For more-or-less random reasons, I decided to look at effective rates over the years. After all, what with all the yelling and screaming about taxes over the past thirty years, you’d think they’ve been wildly gyrating. Not so:

I picked data for the top quintile of people, figuring that you, dear reader, were most likely to fall into that category.

Now, changes in tax rates aren’t the only way people’s incomes change. So I decided to plot three lines:

• The after-tax income of the prototypical top-quintiler.

• The after-tax income assuming that, starting with 1979, effective tax rates never increased. That is, every decline to a new low was allowed, but not the reverse. So the effective rate remained at its 1986 low of 23.8% until 2003 (the last year for which I could find data).

• The after-tax income assuming the reverse: that only increases happened. So the rate remained at 27.5% until the 1995 increase to 27.8% and got pegged at 28% the next year.

The result:

When it comes to taxes, difference between a Reagan and a Clinton is just not that huge. Choosing who to vote for based on tax zealotry is probably silly. Lots of other things the government does has a larger effect on your income and your children’s.

I could be missing something.

For some time now, I’ve been skeptical of the ROI of end-to-end automated tests and of the value of automating the kind of business-facing examples that drive development.

I’ve walked the walk. The Critter4Us app that’s being used at the University of Illinois vet school does not have these kinds of tests. I’m doing contract programming on another app. I make heavy use of Growing Object-Oriented Software-style tests, but I don’t have any that are larger than unit-sized.

What I’ve discovered with Critter4Us is this: if I do what I consider good TDD, then I run through end-to-end tests by hand, and follow up with some not-wonderful exploratory testing*, I do not have bugs that escape to production but would have been caught by full end-to-end tests. (* It’s not-wonderful exploratory testing because I’m a not-wonderful exploratory tester.)

I have written some partial end-to-end tests that exercise the route through the server from HTTP Request to HTTP Response*. Even those are probably not justifiable if the question you care about is “Do they find bugs manual testing would have found, only faster?” However, I write them for two reasons. First, they make me feel better about the pacing of my work, and my own ease-of-work is important to me. The second reason is that I believe a lot of progress in Agile has come through people wanting something, being so naíve they didn’t realize it was impossible for them to have it, so they changed their context to make it possible. So I’m edging toward writing end-to-end tests as a way to force myself to figure out how to make them cost-effective to me.

(* These are very partial end-to-end tests because most of the code lives in the browser front end.)

However, these apps—while “real”—are relatively small, and I do see occasional tweets saying “Having those automated end-to-end tests really saved our butts today!” I’d like to examine some of those bugs in detail so that I can (preferably) discover what kinds of bugs make end-to-end tests worthwhile and thus what specific kinds of end-to-end tests are worthwhile or (less exciting) figure out what unit-style tests were missing.

So email me if you have a juicy bug. But please be aware that “in detail” likely means NDA-level detail and possibly a fair amount of email back-and-forth. And I will want to describe the bugs and systems (in sanitized form) to a worldwide audience.

With one major exception, I’m one of those “if your code needs documentation, it’s not written right” people. The exception is code that answers the question “Why do that instead of this other obvious thing?” If I’m answering a “why” question with a comment, I always wonder whether the comment should go in the code or in the tests.

As an example, consider an app I’m working on. In response to a user request (picking a menu item), a potentially long-running network transfer starts. What the user sees while that’s going on is that the app switches to the tab that normally displays the table of data she’s asked for, that table has been erased, and there’s a progress indicator spinning there.

Getting that to work is more complicated than you might think. Let’s pretend you’re trying to understand it.

The code behind the menu item posts a notification:

The DirectoryController’s initialization declared that this method gets that notification:

The RemoteDirectory also fields the notification. It looks like this:

First question: do you think the comment is helpful? If not, should I also explain that the Savon SOAP library doesn’t (seem to) allow asynchronous operation? Should I explain that RubyCocoa doesn’t allow threading, so I can’t invent asynchrony myself?

Second question: Would you rather see the comment here, in the code, or in the tests? Here’s the test of the load method:

Does the test name reduce or eliminate the need for the comment? (I would normally be using shoulda and assert2.0 here. I’m not because the project was originally in MacRuby, which didn’t handle them at the time I switched to RubyCocoa.)

If you think the name helps, does that suggest I should change the name in the product code to something other than load? What?

James Shore said something in two tweets that I’ve abbreviated as follows:

I’ve never liked mock objects. At best, they’re a necessary evil. At worst, a meaningless “test” that overspecifies the code under test.

I tweeted back that “I’m finding mocks that overspecify the code under test are a sign that I’m making code that doesn’t express intent well.”

Here’s something of what I mean by that. Suppose I want to describe, in English, what it means for a function to make an X509 certificate from some Base64-encoded text and store it in a particular file. The method is part of a Contact object. Here’s the description:

Contacts, as you know from an earlier description [test], are created with a pile of information contained in a hash. One bit of relevant information is the :organization. There are also some Base64-encoded keys. Let’s pick one at random, call it :a_key.

In order for the Contact to create and stash the certificate, it collaborates with two objects. One of them is the :certificate_maker; the other is a :folder that knows how to stash info into a named file. The :certificate_maker is hardwired into the Contact, but the :folder is passed in.

In order to stash :a_key in a :folder, the :stash_this_key method has to do two things:

1. It has to hand the (Base64-decoded) certificate info (the key) to the :certificate_maker, which will return some sort of encoded gibberish.

2. The :folder has to be told to stash the gibberish into one of its files. The filename is of the form organization-name.a_key.pem. (The organization name is made according to the rules I already told you for sanitizing organization names.)

Here’s the actual test. I think you can see how it parallels my description:

It does specify implementation. Most importantly, it says that there should be a completely separate CertificateMaker object. That’s a good thing. Had I not been committed to mocking, I would have been tempted to just copy-and-paste the code I was working from, which looked something like this:

Had I done so, I would have likely never bothered to figure out what sort of magic thing "openssl x509 -inform der" does. As it was, I had to know enough to make a decent class and method name, meaning I had to learn my domain.

Given that the hard part of object-oriented design is figuring out what objects should exist and how they should interact, something that forces me to think harder about interactions and responsibilities might well be a win.

That said, there’s a constant temptation to make the implementation “incidental”, to describe what does happen in this implementation rather than what ought to happen in any reasonable implementation that fits within the, uh, conceptual domain of the program. For example, Ruby already has abstractions for file systems: the Dir and File classes. If my mock tests for Contact talked about how File and Dir are called, … that would be bad. Instead, test expressiveness forced me to condense down everything Ruby lets me do into the more limited idea of a PreferredFolder that knows how to “stash” strings.

I grant you that I still feel like I’m going overboard when I start with 27 lines of sample code and later discover I’ve got a RemoteDirectory and a Directory and a Contact and a CertificateMaker — all just to snatch something off the network, convert it, and stuff it into a file. Certainly, I’m going slower than I could (in the early part of the project), but I don’t necessarily think that’s a bad thing. I trust that it’ll keep the cost of change curve flatter — though my trust in myself might be misplaced. Wouldn’t be the first time.

It took me quite a while to figure out a comfortable way to do my now-comfortable style of mock-heavy TDD while using MacRuby, Xcode, and IB, so I thought I’d boil that work down into a skeleton others can reuse. Enjoy.

It would begin like this:

On a bright Illinois morning on the sixth of November, 2010, Teresa Williams dragged a picture of her twins in their Halloween costumes into a mail message. She clicked a button to send it to their grandparents. Three minutes later, in Göteborg Sweden, Teresa’s mother’s phone buzzed discretely. The mail had arrived.

This book is about everything that happened between the click and the buzz.

It would be a popularization, hitting a technical level somewhere around Singh’s The Code Book—that is, technical enough that someone who really knows the details wouldn’t consider it incorrect, but accessible to the layperson.

What do you think?