Exploration Through Example

[This is a combination of what I intended to say at XP Day Toronto last weekend and what I did say. Note: After the talk, a couple of people told me that Kent Beck has some videos that make some of the same points. I haven’t looked at them yet.]

[UPDATE: INFOQ has an overview and better link to the Beck videos.]

First, a disclaimer: Although I was one of the authors of the Manifesto for Agile Software Development, don’t think that gives me any great authority. If I remember right, my only contribution to it was suggesting the word “manifesto.” Nevertheless:

When reading anything, it helps to keep two questions in mind: What’s the intended audience? And what stance is the author taking? I believe that the stance of the Manifesto is that of a group of development team members trying to explain themselves to those who fund software projects, with the aim of getting those funders to let the teams do this weird thing they’d just dubbed “Agile.”

Despite coming out for customer collaboration over contract negotation, the Manifesto is something of a contract proposal: We promise to give you working software (at frequent intervals, no less), and we promise to stop whining when you change your mind late in the project. In return, you need to stop slowing us down with things like documents and tools and processes that are sometimes intended to be helpful (though not actually helpful) and sometime are ways to keep tabs on us because you don’t trust us. Instead of watching us indirectly, you need to come talk to us. A lot.

That was an appropriate and successful message for the time, but it’s now a different time. Now Agile is more respectable, a safer choice. The challenge isn’t so much getting a chance at Agile as it is executing once you’ve gotten the chance. There, the values of the Manifesto are not so helpful. Except for “individuals and interactions”, they face outward from the team. They don’t strongly apply to the relationships between team members, between the teams and their environments, and between the teams and their code.

That’s a problem, because it seems to me that many new Agile teams are not executing. They’re floundering. [Here, I talked about teams starting out too fast, as I did earlier in this blog.]

What should teams do with the time they’re not spending going too fast? They should invest in one of the four values I want to talk about: skill. As someone who wants to remain anonymous said to me:

I’ve also been tired for years of software people who seem embarrassed to admit that, at some point in the proceedings, someone competent has to write some damn code.

He’s a programmer. As a tester-programmer blend, I’d want to also say that someone competent has to test the damn code, write the damn user documentation, and deploy the damn thing.

Two skills that apply to pretty much any software project are refactoring and programmer testing (test-driven design). Those are skills that require a great deal of discipline. It’s awfully tempting not to write a test when it’s harder than writing the code, or to refactor some icky code you stumble on when that would mean blowing your estimate. Some of XP’s practices help with discipline. Pair programming turns what could be a solitary vice into a social act: you and your pair have to look at each other and acknowlege that you’re about to cheat. Peer pressure comes into play, as it does because of collective code ownership. Someone will notice the missing tests someday, and they might know it was your fault.

Maybe the key driver of discipline is the practice of creating working software—running, tested features—at frequent intervals. If you’re doing that, you just don’t have time for sloppiness. You have to execute well. I call this the lash of Jeffries, since Ron Jeffries talks a lot about it.

But wait: if the business is whipping us with the Lash of Jeffries, and the Lash requires good execution, why do so many projects go too fast and too sloppily? Well, I have to reveal a fact that will surprise no one but a University of Chicago Economist:

Business people often don’t care that much about business value.

They may find political gamesmanship a more suitable move in the great primate game of status that we all play to some extent or another. Or early childhood trauma may have left them with a command-and-control nature. In that case, business value may be sacrificed at the altar of their inner demons. Or they may simply not have enough time; something has to give; and customer collaboration, done well, consumes enormous amounts of time. Whatever the reason, they spare the lash and spoil the team.

This is a big problem, I think, because it means we have to find some way of supplying discipline to replace the Lash. We don’t seem to be so good at it.

Let me shift topics. How can you tell when you’re applying skill well?

Richard P. Gabriel noticed, some years ago, that we can think of programmers inhabiting code kind of like human beings inhabit houses. Maybe the properties of a nice house to live in carry over to the program and programming environment and tools. (See the online PDF of his book, Patterns of Software.)

If you snoop around my house, you’ll notice that there’s a silverware drawer that’s rather tidy and a junk drawer that’s not. It’s much easier to find a spoon than a battery. There’s a reason for that: my family and I look for spoons several times a day, but for batteries much less often. Things you use often should be easy to find. Things you do often should be easy to do. Properties like this, which Gabriel collectively calls “habitability,” can go against software principles. For example, a gas stove in the ground floor kitchen unnecessarily couples the kitchen to the laundry room in the basement, with its gas dryer. It would be a cleaner design to put both the stove and the dryer in the same place. Except that would be stupid. Our comfort in cooking overrides saving twenty feet of pipe.

Another aspect of habitability is that houses reward tinkering. To the left, you can see a picture of my office.

As you can tell, I’m not much on aesthetic workspaces. So it won’t be a problem when I hang Power Squids from the ceiling to make it easier to plop a new disk drive on the desk and give it power. How convenient: two anchors, two holes, and two hooks later, I’ll have made an easy change that makes the house a bit more livable.

It’s also possible, within constraints, to make major changes to a house. The cover to Brand’s fine How Buildings Learn shows two initially identical buildings that have diverged over the years:

Houses are surprisingly malleable. In some places it wasn’t unusual to put in the basement after the house was built, something that seems like the very epitome of too-difficult-to-bother-with.

(Most) houses are made to make us comfortable. Our software can be that way, too. I’ve observed that a characteristic of a good code base is that it’s habitable, that changes are comfortable, that the process of programming is one of ease. Agile teams suffer because they don’t think that wanting to make their lives easy is a relevant value, but it should be. There’s absolutely nothing wrong with finishing up a story by spending a half hour or so tinkering with code you’ve touched, just improving it—doing the software equivalent of hanging power within easy reach because the power strip on the desk is always annoying you.

So, suppose teams embraced skill, discipline, and ease. Would all be well? No: we’re a cussed species. Like any other pack animal, we spend a lot of time butting heads over status. Me too, but I’m getting better. One person who helped me get better was Ron Jeffries. On a joint consulting gig, we sometimes paired right near each other. I noticed he handled design discussions differently than I did. When my pair and I had different ideas about which direction to take, we’d often talk about it for a good while, trying to reach agreement on the right way. It seemed that Ron and his pair talked less, and the talk often ended up with Ron saying, “Let’s try it your way.”

When I asked Ron about it, he explained that, while he thought his way was right, maybe he was wrong. In that case, taking the other path would teach him something. Even if he was right and they started off down the wrong path, it was no big deal. They could always bend the path in the right direction, likely without much trouble at all. (Remember: software can be soft.) I’ve since adopted his practice, and it works.

Pause for a moment. How often do you hear someone advising you to do the wrong thing, just because it makes someone else’s life pleasant? But that’s exactly what I’m advising.

Let me give you a different example. It’s common for programmers to have spiffy new hardware while the testers make do with crummy, cast-off PCs. There’s often something to be said for that: those PCs are probably more representative of what the end users use. And the testers don’t really need so much power for what they do. To that, I can only reply as Lear did when his daughters told him he didn’t really need all those knights:

O reason not the need! Our basest beggars
Are in the poorest thing superfluous.
Allow not nature more than nature needs,
Man’s life is as cheap as beast’s.

If the testers would be happier with better PCs, just because it makes them feel respected, why not give them what they want? Why not make people happy? Why make their lives as cheap as beast’s? And why just sit there in a structure that treats your life as cheap as a beast’s, that treats you as nothing more than a fungible economic unit?

This matters because many of the original advocates for Agile became so because of that one great project, the one that makes the awkwardness and nastiness and pointless unpleasantness of all the others seem a blight that has to be wiped out. On that project, they found joy, and the Agile movement is about spreading it around.

Now, I could say that a joyful employee is a productive employee, and that lack of joy on a project is like a canary keeling over in a coal mine: a sign that something big is wrong and you better pay attention. Maybe that’s true. I’d certainly like to believe it. But, fundamentally, I don’t care. I think joy is its own excuse. We deserve it. More to the point, those people around us deserve it. Their work shouldn’t suck any more than yours does, and you have it easily within your power to remove some suckitude.

As I said at the beginning, I think Agile is suffering today because these fundamental values didn’t get written down and are too easily forgotten. As Agile moves into bigger companies and into less adventurous ones, those undocumented values are getting watered down. If that continues, I’m afraid Agile will be this decade’s fad that changes nothing. That would be sad.

I’d like to end this talk with a stirring conclusion, a call to arms that pulls you out of your seat. I’d like to end all my talks that way, but I’m no good at conclusions. So I’ll just stop.