Exploration Through Example

Example-driven development, Agile testing, context-driven testing, Agile programming, Ruby, and other things of interest to Brian Marick
191.8 167.2 186.2 183.6 184.0 183.2 184.6

Fri, 14 Mar 2003

Usability bugs and exploratory testing

Laurent Bossavit tells of a usability bug his wife encountered. It seems a library checkin system produced an annoying penalty screen, with sound, for each overdue book. When Laurent's wife checked in thirty books, the penalty beeps drove the librarian mad. Laurent asks "Could a tester have seen this one coming?"

An interesting question. Suppose I were doing automated product-level testing. I would certainly have a test in which I checked in multiple overdue books. (I'd probably toss in some not-overdue books as well.)

That is, I would not at all be surprised to start typing a test like this:

  def test_big_checkin_with_fines
    // add code to create some books

    @terminal.scan(three_days_overdue_book)
    assert_overdue_notice(@terminal)
    @terminal.acknowledge
    assert_checked_in(three_days_overdue_book)

    @terminal.scan(within_grace_period_book)
  

At about that time, I'd get annoyed because I could see a bunch of redundant typing coming up. So I'd extract a utility:

  def assert_overdue_checkin(book)
    @terminal.scan(book)
    assert_overdue_notice(@terminal)
    @terminal.acknowledge
    assert_checked_in(book)
  end
    
  def test_big_checkin_with_fines
    // add code to create some books

    assert_overdue_checkin(three_days_overdue_book)
    assert_overdue_checkin(within_grace_period_book)
    assert_overdue_checkin(ten_days_overdue_book)
    @terminal.checkin(actual_on_time_book)
    assert_checked_in(actual_on_time_book)

    ... check that the fines were calculated correctly ...

  end

More intention-revealing, though there is room for improvement. Much more pleasant to type. In fact, I've factored out the annoyance that, in a more audible form, is the bug (using Bret's definition of a bug as "something that bugs someone").

It's my belief that the act of automating would make me pretty unlikely to stop and say, "You know... this repetition is really annoying."

I further believe that those sorts of annoyances tend to skate past end-of-iteration demonstrations and the trials that product managers or agile-style Customers make. It's only the real users who use things enough to have their nerves rubbed raw. Real users... and manual testers. Manual testers are the only people on most teams who use the product like the real users and anywhere near as much as the real users.

That's one of the reasons why, when I talk about agile testing, I swim a bit against the current and hold out a role for manual exporatory testers.

P.S. "Nail-Tinted Glasses" (couldn't determine the real name) points out that interface design people ought to have the experience to avoid the problem. Agreed. They, like testers, should learn about bugs from experience. But, when they slip up, who notices?

## Posted at 16:55 in category /testing [permalink] [top]

About Brian Marick
I consult mainly on Agile software development, with a special focus on how testing fits in.

Contact me here: marick@exampler.com.

 

Syndication

 

Agile Testing Directions
Introduction
Tests and examples
Technology-facing programmer support
Business-facing team support
Business-facing product critiques
Technology-facing product critiques
Testers on agile projects
Postscript

Permalink to this list

 

Working your way out of the automated GUI testing tarpit
  1. Three ways of writing the same test
  2. A test should deduce its setup path
  3. Convert the suite one failure at a time
  4. You should be able to get to any page in one step
  5. Extract fast tests about single pages
  6. Link checking without clicking on links
  7. Workflow tests remain GUI tests
Permalink to this list

 

Design-Driven Test-Driven Design
Creating a test
Making it (barely) run
Views and presenters appear
Hooking up the real GUI

 

Popular Articles
A roadmap for testing on an agile project: When consulting on testing in Agile projects, I like to call this plan "what I'm biased toward."

Tacit knowledge: Experts often have no theory of their work. They simply perform skillfully.

Process and personality: Every article on methodology implicitly begins "Let's talk about me."

 

Related Weblogs

Wayne Allen
James Bach
Laurent Bossavit
William Caputo
Mike Clark
Rachel Davies
Esther Derby
Michael Feathers
Developer Testing
Chad Fowler
Martin Fowler
Alan Francis
Elisabeth Hendrickson
Grig Gheorghiu
Andy Hunt
Ben Hyde
Ron Jeffries
Jonathan Kohl
Dave Liebreich
Jeff Patton
Bret Pettichord
Hiring Johanna Rothman
Managing Johanna Rothman
Kevin Rutherford
Christian Sepulveda
James Shore
Jeff Sutherland
Pragmatic Dave Thomas
Glenn Vanderburg
Greg Vaughn
Eugene Wallingford
Jim Weirich

 

Where to Find Me


Software Practice Advancement

 

Archives
All of 2006
All of 2005
All of 2004
All of 2003

 

Join!

Agile Alliance Logo