Exploration Through Example

Clojure.zip is a library that lets you move around freely within trees and also create changed copies of them. This is a tutorial I wish I’d had when I started using it.
Read the rest of this entry »

Earlier today, I found myself talking to Andrew Shafer and Corey Haines about certification. Certifications provided by people who make good money off them beg for distrust. So do certifications of courses or trainers when the certifiers get revenue from that activity. On balance, I think we’d have been better off without the certifications we have, and I agree with the Agile Alliance’s position on certification. (Disclosure: I was the main author.)

However. Here we are, aren’t we?

To improve the current situation, I propose the following, roughly inspired by Consumer’s Union.

• There should be an Agile Alliance program that applies to all courses, not just those that provide certifications.

• This program will, with the Agile Alliance’s money, send spies to sign up for courses. They will anonymously write up detailed evaluations that will be posted on the Agile Alliance website.

• It will also interview a random sample of course attendees. The ones contacted just after the course will be asked for their evaluation. The ones contacted a year after the course will be asked how valuable, in retrospect, the course was. These results will also be posted on the site.

• These interviews and ratings will, over time, allow the writing of summary articles like the ones Consumer Reports Magazine runs. Expect articles about “What to Expect from a Certified ScrumMaster” or “How to select a TDD course”.

• Those running the program will pay careful attention to maintaining the trust of their readers.

It’s possible I’d participate in this. I have no desire to run it. I don’t care that much.

While working on the “sweet” interface for Midge, I wrote this function:

metavar? and form-branch? are only useful within this function. I would have preferred to write it like this:

I didn’t because I wanted those two functions tested before I relied on them. So I need a format for testing inner functions. I’ve sketched out a format and done (by hand) the transformation required to make it work.

I’ll start explaining it with this simple example:

Before dealing with the daunting complexity of multiplication, I want to test summer with something like this:

(I’m using the Midje sweet notation here. Read it as “It’s a fact that (within (outer 1 2)), (summer) produces 3.”)

To make this work, I’d redefine the defn macro for the duration of a test and have it produce a little extra metadata:

I’m stashing away a function derived from summer. It creates the same lexical environment that outer does, and returns a function to call. Here’s the macro that would expand a within, together with a sample expansion:

The expanded form is a little opaque, so here are the three steps:

1. Retrieve the environment setting function from outer’s metadata.
2. Call it with the outer arguments.
3. Within that environment, call summer.

What if a later inner function uses a previously-defined one? Like this:

One option would be to drag the earlier parts of the let into the metadata:

In that case, the test would look like this:

This test is a little not-thrilling because it’s clearer what multiplier does if you can see how it relies on summer. That could look like this:

However, dragging the previous let clauses into the metadata has to be done anyway (in case non-function values are bound), so I’d be inclined not to gild this lily.

Creating the metadata wouldn’t be too hard in the common case where the let is the outermost form in the function’s definition. I doubt I could be persuaded to care about other cases.

Michael Bolton has asked me to improve the sentence in the headline. Here is my attempt. It stems from two of my beliefs:

• It’s better to focus on the immediate problem than on convincing people of a universal truth.

• Right action does not follow from thinking correctly. Thinking correctly follows from the habit of acting correctly.

The sentence is an aphorism: terse, stylish, true, and deep. I think its proponents value the aphoristic quality as much as they do the underlying sentiment. When I watch them use it, I believe they’d be happiest if they could generate an epiphany (”a sudden, intuitive perception of or insight into the reality or essential meaning of something”). I’ll try and mimic some of the aphoristic terseness at first, even though I am explicitly not going for an epiphany.

Here goes:

1. The setting: an unproductive conversation needs to be redirected. It’s likely that the conversation is happening because some important people are unhappy. But instead of talking about how to make them happy, we’re talking about whether they are right to be unhappy. We’ve come down to—explicitly or implicitly—competing definitions of quality.

2. There is a brief lull in the conversation. Our hero speaks.

“Different people care differently.” (beat)

Because the sentence doesn’t obviously have anything to do with quality, it has a desirable “huh? where did that come from?” effect that keeps people from jumping into our hero’s pause-for-emphasis.

3. The hero speaks the following sentence with an “opening up / giving” hand movement, the sort where you bring your hand up and out, rotating it from palm down to palm up. This counters the harshness and abruptness of the first sentence: Diff-rent pee-pul judge diff-rent-ly.

“So different people will judge our product differently.” (beat)

A little rhetorical flourish in the parallelism of the two sentences. The “so” at the beginning switches the meter to more iambic, gives it a little forward momentum, opens it up into a more conversational tone. “Judge” is a strong, active word (as is “care”), and focuses attention on what people are doing that the group must react to.

4. Someone tries to jump into the pause. Our hero gains time for one more utterance with a “just one more thing” head-tilt-back, audible intake of breath, and soft palms-out rotation of the hands, forefinger up.

5. The next bit takes long enough to give our hero a chance to make contact, via hand and head movements, with the people who are bored sick of the argument and have been sitting silently while the true believers battle it out. They’re being invited to break their silence to lend support.

“I predict that if we figure out which people we care about, and figure out what they care about, and how we can know their judgments before we get surprised like we were, we can live long and happy lives without ever agreeing what quality is. Let’s give it a try, OK?”

Long—because the problem isn’t simple. A program for action. Inviting a group of arguers to become “we” again.

The whole structure of the three parts is moving from authoritarian (”a fact you must recognize is…”) and abrupt to light (”long and happy lives”) and consensus-seeking (”we”, “OK”, ending on a rising question-tone).

I’ve long hated the sentence in the title, for a number of reasons. I threw away a long blog post about that yesterday, but let me just issue this challenge. How do the meanings of these three sentences differ?

Quality is value to some person.
Quality is quality to some person.
Value is value to some person.

That’s the title of my Agile2010 workshop. Here’s the description:

We’re not shy about saying that testers, programmers, and everyone else in the team needs to become fluent in the business domain. But we’re blasé when programming practices and programming knowledge are the exclusive province of the programmers. That hampers communication and hurts teams. By simply having people stand up, move around, and speak messages to each other, this workshop will show anyone who attends how advanced object-oriented programming and test-driven design works. Although tailored to non-programmers, programmers may find it useful too.

The devoted reader will recall that I earlier sketched a “little language” for mocking Clojure functions. I’ve finished implementing a usable–perhaps even pleasant–subset of it. I invite you to take a look. In keeping with my whole “work with ease” thing, I’ve made it dirt simple to try it out: one click and two shell commands. You don’t even have to have Clojure installed.

It’s often the case that a class has one or more collaborators that are used by many of its methods. When you want to override one for testing, you can either pass the test double into the constructor (probably as a default argument) or you can smash the double into an instance variable you happen to know points to the collaborator.

I used to use the constructor approach, but I increasingly find myself using the Hulk! Smash! approach. It looks like this…

The object-to-be-tested names its collaborators and gives them the values they’ll use in production:

I do this because I think of the collaborators as a static, declaration-like property of the class. (If I were ambitious, I’d convert collaborators_start_as into a class-level declaration like this:

I haven’t been that ambitious yet, as it turns out.)

Each test declares which collaborators it overrides:

Note that (in Ruby), I can even mock out classes, so I don’t have to go through contortions to pass in instances that an outside caller really would have no particular interest in:

With several of the mocking packages available to me, I wouldn’t even have to go through the indirection of putting the class ThingSource into an instance variable. I could do this:

I’m not bold enough to do that yet.

Various Republicans have characterized the recently announced 20 billion dollar deal between the government and BP as a “shakedown” for a slush fund, one that Obama doesn’t have constitutional authority to set up. From what I’ve read, it strikes me as a pretty straightforward deal, not wildly different than the way I used an escrow company as an independent party when selling testing.com. Or that unpleasant incident with my neighbor’s grill a few years back…

Me: You careless oaf! You burned up my lawn!

Him: I admit it. I take full responsibility. Sorry.

Me: You’re going to pay for this damage, you know. All of it.

Him: I will do that.

Me: I bet that big old maple that overhangs the house is going to have to come out. That’s going to cost you a bundle.

Him: Oh now, it doesn’t look that bad.

Me: I had a tree hit by lightning once. Didn’t look much worse than that, but carpenter ants got into it and the tree had to come out. The guys who took it out told me I should have done it right away, that the tree was worse off than it looked. I’m not waiting this time.

Him: Uh… Don’t know all that much about trees.

Me: Neither do I. Look. Let’s simplify this, keep at least some of it out of court. Let’s get someone with experience at judging these kinds of damages and have him decide, shrub by shrub, tree by tree, what you owe.

Him: Like the independent mediator written into lots of contracts.

Me: Right.

Some time passes as we dicker over who we both trust.

Me: Now, I’m pretty sure this is going to cost you over $2000…

Him: Aw, that seems high…

Me: That maple is four stories tall!

Him: Hey, I still think it’ll be alright. But OK—so long as we get that mediator guy, the one I trust.

Me: And I want you to put the money in escrow.

Him: Aw, c’mon. You know I’m good for it.

Me: Yeah, well, my trust in you is not super-high right now.

Him: Well…

Me: Look. Back out of the deal if you want. You’re going to pay one way or the other. You can do it quickly or drag it out. Your choice.

Him: OK, OK: it’s a deal.

Part 1
Part 2

All that remains is to add the locations of bordering cells to a given sequence of locations. A small wrinkle is that a border location may be next to more than one of the input locations, so duplicates need to be prevented. I could write the test this way:

I don’t like that. When the test fails, it’ll likely be hard to discover how the expected and actual values differ. And I bet that a failure would more likely be due to a typo in the expected value than to an actual bug. The test is just awkward.

I like this version better:

It is a clearer statement of the relationship between two concepts: a location’s neighborhood and the border of a set of locations. More pragmatically, it’s less typing. (When I first started coding in this style, it surprised me how much test setup clutter went away. I had much less need for “factories” or “fixtures” or “object mothers“.)

Given either test, the code is straightforward:

Onward

No matter how satisfying the individually-tested pieces, the whole has to work, which is why everything ends by running at least one end-to-end test. A test like the one we started with:

Because I’m making up my test notation as I go, I’ve been running all the tests manually in the REPL. Now I can run the whole file:

If you look carefully, you’ll see that the test would fail because the locations are in the wrong order. But that’s a quick fix:

Done. Ship it!

Development order - Test-down or REPL-up?

It’s often said that the Lisps are bottom-up languages, that you test out expressions in the REPL, discover good functions, and compose them into programs. A lot of people do work that way. A lot of people who use TDD to write object-oriented code also work that way: when implementing a new feature, they start at low-level objects, add whatever new code the top-level feature seems to demand of them, then use those augmented objects in the testing of next-higher-level objects.

For I guess about a year now, I’ve been experimenting with being strictly top-down in some projects. I find that leads to less churn. Too often, when I go bottom-up, I end up discovering that those low-level changes are not in fact what the feature needs, so I have to revisit and redo what I did.

I get less disrupted by rework when I go from the top down (or from user interface in). It’s not that I don’t blunder—we saw one of those in the previous installment—but those blunders seem easier to recover from.

That’s not to say that I don’t use the REPL. We saw that a little bit in this program, when I was writing neighbors. It’s perfectly sensible to do even more in the REPL. I think of the REPL as a handy tool for what XP calls a spike solution and the Pragmatic Programmers call tracer bullets. When I’m uncertain what to do next, the REPL is a tool to let me try out possibilities. So I might be stuck in a certain function and go to the REPL to see how it feels to build up parts of what might lie under it. After I’m more confident, I can continue on with the original function, test-driven, reusing REPL snippets when they seem useful.

I don’t claim everyone should work that way. I do claim it’s a valid style that you’d be wise to try.

Notation

I’m something of an obsessive about test notation, and I’ve been endlessly fiddling with a Clojure mock notation. I implemented its first version as a facade on top of clojure.contrib.mock. As I experimented, though, I found that keeping my facade up to date with my notational variants slowed me down too much, so I put the code aside until the notation settled down.

I’m pretty happy with what you’ve seen here. Are you? If so, I may start on another mock package. I’ve got the most important parts: a name and sketch of a logo. (”Midje” and someone flying safely between the sun [of abstraction without examples] and the sea [of overwhelming detail].)

Tests and code together

You can see the completed program here. It mixes up tests and code. I’ve tried that on-and-off over the years and always reverted to separate test files. This time, it’s seemed to work better. I probably want an Emacs keystroke that lets me hide all tests, though. I’d also want alternate definitions of the test macros so that I can compile them out of the production system

What next?

I’ll write a web app in this style, using Compojure.