Exploration Through Example

I’ll be giving one of the keynotes at XP2011 (Madrid, May 10-13). Here’s my tentative abstract. How do you like it?

What Forms of Work-and-Life Make Sense for Us?

It’s widely agreed that Agile has “crossed the chasm” to mainstream acceptance. Along the way, some of the more interesting bits have fallen off. This talk will be the latest in a multi-year effort to recover those bits, rehabilitate them, make them stranger, and encourage you to put them to work. There will be no overarching “Big Idea”, but some smaller ideas will include the distinction between what I call “the stance of rationality” and “the stance of reaction” (I favor the latter), the high cost of thinking, the virtues of habit, and some criticism of the optimistic ideas about teams, enterprises, and leaders that infest our field. There may also be a tango lesson.

Building on the foundation of the Identity Decider from Part 1, I first introduce some notation to make a monadic computation look more like a `let`. Then I explain the Maybe monad and what it tells us about how monads are meant to be used. Finally, I show what the Sequence monad does as a teaser for Part 3.

http://vimeo.com/20798376

For Midje, I need to take a data structure and produce a new data structure that’s = to the old one, but with all lazy sequences converted into lists. It was hard to get it right. I’m posting my solution here in the hopes that someone’s search engine saves them some work.

Earlier today, I found myself talking to Andrew Shafer and Corey Haines about certification. Certifications provided by people who make good money off them beg for distrust. So do certifications of courses or trainers when the certifiers get revenue from that activity. On balance, I think we’d have been better off without the certifications we have, and I agree with the Agile Alliance’s position on certification. (Disclosure: I was the main author.)

However. Here we are, aren’t we?

To improve the current situation, I propose the following, roughly inspired by Consumer’s Union.

• There should be an Agile Alliance program that applies to all courses, not just those that provide certifications.

• This program will, with the Agile Alliance’s money, send spies to sign up for courses. They will anonymously write up detailed evaluations that will be posted on the Agile Alliance website.

• It will also interview a random sample of course attendees. The ones contacted just after the course will be asked for their evaluation. The ones contacted a year after the course will be asked how valuable, in retrospect, the course was. These results will also be posted on the site.

• These interviews and ratings will, over time, allow the writing of summary articles like the ones Consumer Reports Magazine runs. Expect articles about “What to Expect from a Certified ScrumMaster” or “How to select a TDD course”.

• Those running the program will pay careful attention to maintaining the trust of their readers.

It’s possible I’d participate in this. I have no desire to run it. I don’t care that much.

Michael Bolton has asked me to improve the sentence in the headline. Here is my attempt. It stems from two of my beliefs:

• It’s better to focus on the immediate problem than on convincing people of a universal truth.

• Right action does not follow from thinking correctly. Thinking correctly follows from the habit of acting correctly.

The sentence is an aphorism: terse, stylish, true, and deep. I think its proponents value the aphoristic quality as much as they do the underlying sentiment. When I watch them use it, I believe they’d be happiest if they could generate an epiphany (”a sudden, intuitive perception of or insight into the reality or essential meaning of something”). I’ll try and mimic some of the aphoristic terseness at first, even though I am explicitly not going for an epiphany.

Here goes:

1. The setting: an unproductive conversation needs to be redirected. It’s likely that the conversation is happening because some important people are unhappy. But instead of talking about how to make them happy, we’re talking about whether they are right to be unhappy. We’ve come down to—explicitly or implicitly—competing definitions of quality.

2. There is a brief lull in the conversation. Our hero speaks.

“Different people care differently.” (beat)

Because the sentence doesn’t obviously have anything to do with quality, it has a desirable “huh? where did that come from?” effect that keeps people from jumping into our hero’s pause-for-emphasis.

3. The hero speaks the following sentence with an “opening up / giving” hand movement, the sort where you bring your hand up and out, rotating it from palm down to palm up. This counters the harshness and abruptness of the first sentence: Diff-rent pee-pul judge diff-rent-ly.

“So different people will judge our product differently.” (beat)

A little rhetorical flourish in the parallelism of the two sentences. The “so” at the beginning switches the meter to more iambic, gives it a little forward momentum, opens it up into a more conversational tone. “Judge” is a strong, active word (as is “care”), and focuses attention on what people are doing that the group must react to.

4. Someone tries to jump into the pause. Our hero gains time for one more utterance with a “just one more thing” head-tilt-back, audible intake of breath, and soft palms-out rotation of the hands, forefinger up.

5. The next bit takes long enough to give our hero a chance to make contact, via hand and head movements, with the people who are bored sick of the argument and have been sitting silently while the true believers battle it out. They’re being invited to break their silence to lend support.

“I predict that if we figure out which people we care about, and figure out what they care about, and how we can know their judgments before we get surprised like we were, we can live long and happy lives without ever agreeing what quality is. Let’s give it a try, OK?”

Long—because the problem isn’t simple. A program for action. Inviting a group of arguers to become “we” again.

The whole structure of the three parts is moving from authoritarian (”a fact you must recognize is…”) and abrupt to light (”long and happy lives”) and consensus-seeking (”we”, “OK”, ending on a rising question-tone).

That’s the title of my Agile2010 workshop. Here’s the description:

We’re not shy about saying that testers, programmers, and everyone else in the team needs to become fluent in the business domain. But we’re blasé when programming practices and programming knowledge are the exclusive province of the programmers. That hampers communication and hurts teams. By simply having people stand up, move around, and speak messages to each other, this workshop will show anyone who attends how advanced object-oriented programming and test-driven design works. Although tailored to non-programmers, programmers may find it useful too.

It’s often the case that a class has one or more collaborators that are used by many of its methods. When you want to override one for testing, you can either pass the test double into the constructor (probably as a default argument) or you can smash the double into an instance variable you happen to know points to the collaborator.

I used to use the constructor approach, but I increasingly find myself using the Hulk! Smash! approach. It looks like this…

The object-to-be-tested names its collaborators and gives them the values they’ll use in production:

I do this because I think of the collaborators as a static, declaration-like property of the class. (If I were ambitious, I’d convert collaborators_start_as into a class-level declaration like this:

I haven’t been that ambitious yet, as it turns out.)

Each test declares which collaborators it overrides:

Note that (in Ruby), I can even mock out classes, so I don’t have to go through contortions to pass in instances that an outside caller really would have no particular interest in:

With several of the mocking packages available to me, I wouldn’t even have to go through the indirection of putting the class ThingSource into an instance variable. I could do this:

I’m not bold enough to do that yet.

One of my pet peeves is how people get all obsessed with marginal tax rates when no one actually pays them. For more-or-less random reasons, I decided to look at effective rates over the years. After all, what with all the yelling and screaming about taxes over the past thirty years, you’d think they’ve been wildly gyrating. Not so:

I picked data for the top quintile of people, figuring that you, dear reader, were most likely to fall into that category.

Now, changes in tax rates aren’t the only way people’s incomes change. So I decided to plot three lines:

• The after-tax income of the prototypical top-quintiler.

• The after-tax income assuming that, starting with 1979, effective tax rates never increased. That is, every decline to a new low was allowed, but not the reverse. So the effective rate remained at its 1986 low of 23.8% until 2003 (the last year for which I could find data).

• The after-tax income assuming the reverse: that only increases happened. So the rate remained at 27.5% until the 1995 increase to 27.8% and got pegged at 28% the next year.

The result:

When it comes to taxes, difference between a Reagan and a Clinton is just not that huge. Choosing who to vote for based on tax zealotry is probably silly. Lots of other things the government does has a larger effect on your income and your children’s.

I could be missing something.

For some time now, I’ve been skeptical of the ROI of end-to-end automated tests and of the value of automating the kind of business-facing examples that drive development.

I’ve walked the walk. The Critter4Us app that’s being used at the University of Illinois vet school does not have these kinds of tests. I’m doing contract programming on another app. I make heavy use of Growing Object-Oriented Software-style tests, but I don’t have any that are larger than unit-sized.

What I’ve discovered with Critter4Us is this: if I do what I consider good TDD, then I run through end-to-end tests by hand, and follow up with some not-wonderful exploratory testing*, I do not have bugs that escape to production but would have been caught by full end-to-end tests. (* It’s not-wonderful exploratory testing because I’m a not-wonderful exploratory tester.)

I have written some partial end-to-end tests that exercise the route through the server from HTTP Request to HTTP Response*. Even those are probably not justifiable if the question you care about is “Do they find bugs manual testing would have found, only faster?” However, I write them for two reasons. First, they make me feel better about the pacing of my work, and my own ease-of-work is important to me. The second reason is that I believe a lot of progress in Agile has come through people wanting something, being so naíve they didn’t realize it was impossible for them to have it, so they changed their context to make it possible. So I’m edging toward writing end-to-end tests as a way to force myself to figure out how to make them cost-effective to me.

(* These are very partial end-to-end tests because most of the code lives in the browser front end.)

However, these apps—while “real”—are relatively small, and I do see occasional tweets saying “Having those automated end-to-end tests really saved our butts today!” I’d like to examine some of those bugs in detail so that I can (preferably) discover what kinds of bugs make end-to-end tests worthwhile and thus what specific kinds of end-to-end tests are worthwhile or (less exciting) figure out what unit-style tests were missing.

So email me if you have a juicy bug. But please be aware that “in detail” likely means NDA-level detail and possibly a fair amount of email back-and-forth. And I will want to describe the bugs and systems (in sanitized form) to a worldwide audience.

With one major exception, I’m one of those “if your code needs documentation, it’s not written right” people. The exception is code that answers the question “Why do that instead of this other obvious thing?” If I’m answering a “why” question with a comment, I always wonder whether the comment should go in the code or in the tests.

As an example, consider an app I’m working on. In response to a user request (picking a menu item), a potentially long-running network transfer starts. What the user sees while that’s going on is that the app switches to the tab that normally displays the table of data she’s asked for, that table has been erased, and there’s a progress indicator spinning there.

Getting that to work is more complicated than you might think. Let’s pretend you’re trying to understand it.

The code behind the menu item posts a notification:

The DirectoryController’s initialization declared that this method gets that notification:

The RemoteDirectory also fields the notification. It looks like this:

First question: do you think the comment is helpful? If not, should I also explain that the Savon SOAP library doesn’t (seem to) allow asynchronous operation? Should I explain that RubyCocoa doesn’t allow threading, so I can’t invent asynchrony myself?

Second question: Would you rather see the comment here, in the code, or in the tests? Here’s the test of the load method:

Does the test name reduce or eliminate the need for the comment? (I would normally be using shoulda and assert2.0 here. I’m not because the project was originally in MacRuby, which didn’t handle them at the time I switched to RubyCocoa.)

If you think the name helps, does that suggest I should change the name in the product code to something other than load? What?