Exploration Through Example
Example-driven development, Agile testing, context-driven testing, Agile programming, Ruby, and other things of interest to Brian Marick
Mon, 19 Dec 2005
Working your way out of the automated GUI testing tarpit (part 4)
The story so far: One of my main goals for tests is that they contain no excess words. That means that a GUI test should not describe the path by which it gets to the page under test. In part 1, I described a declarative format. With it, the test writer specifies all and only the facts that should be true of the app at the point the test begins. Part 2 gives a simple implementation that figures out a path through the app that makes those facts true. Part 3 recommends that you migrate tests to this format only as they fail.
The new format tests, though, run as slowly as they did before being migrated. Now it's time to make them faster. I'll do that in two steps. The first doesn't even double their speed. That's hardly sufficient, but the implementation has a side effect that helps the programmer and exploratory tester.
Previously, I only pretended the app talked across the network. Since that fakery would make any timings useless, it's now running on a real server (WEBrick), fielding real live HTTP. So localhost:8080 shows this stunningly attractive UI:
In part 1, I wrote three versions of a test. All three of them communicate with the server in exactly the same way: they send eight different HTTP GET commands (just as a browser would if you visited the app and then pressed seven buttons on seven pages).
To speed up the test, I've made it remember all eight of the commands the first time it runs. (That all happens behind the scenes; there are no changes to the test.) Now later runs can send the commands in a big glob via a side channel. That avoids seven of the round trips.
The results are underwhelming. The original test takes 5.1 seconds. The version that sends the big glob takes 3.2 seconds. The more complicated the test setup path, the greater the speedup would be, but still—is this worth the trouble?
Not so far, but it will be after the next speedup. I hope. In the meantime, there's a useful spinoff feature. One of the reasons I hate anything to do with improving a UI is that every time you tweak a page, you have to navigate to it to check whether the change looks right. Having to do that four or five times in a row drives me wild. So I wish this were a universal law:
You can get to any page in an app in one step.
Now that we can remember application state, that's possible here. Imagine the following:
You have to tweak a particular page in the UI. You navigate to that page, then type this:
You go into the code, make a change, reload the app, and return the app to its previous state like this:
This jump-to-page feature would also be useful for exploratory testing. It's common to go to the same place in the program multiple times during a bout of exploratory testing. Perhaps you're trying to learn more about the circumstances in which a bug occurs (a kind of failure improvement). Or you're trying different paths through the program, each of which starts some distance into it.
There's nothing new about using captured commands to accelerate tasks. People have been using GUI capture/replay tools for this kind of thing since the dawn of time. But it's nice that the feature fell out of a different goal.
For more about the implementation, refer to part 4b. The code has the complete details.
## Posted at 22:49 in category /testing
Working your way out of the automated GUI testing tarpit (part 4b)
Here are some (decidedly optional) details about the implementation described in part 4.
Consider this test:
The next time
Then each command is dispatched to the App object:
By doing that, I reduce the suspicion that the restored state is somehow different than the one that app had at the moment the snapshot was taken.
The only difference between the two different routes into the app is
what happens after dispatching.
Credit: The idea of replaying server-level commands just popped into my head. It might have been put there by Michael Silverstein's "Logical Capture/Replay".