User Interface Design and Ergonomics - National Open University of ...

ecruit test users with more similar background, and you can try to brief test users to
bring them close to some common level of preparation for their tasks.
Differences in procedure, how you actually conduct the test, will also add to variability.
If you help some test users more than others, for example, you are asking for trouble.
This reinforces the need to make careful plans about what kind of assistance you will
provide. Finally, if people don't understand what they are doing your variability will
increase. Make your instructions to test users and your task descriptions as clear as you
can.
f. Debriefing Test Users
It has been stressed that it is unwise to ask specific questions during a thinking aloud test,
and during a bottom-line study. But what about asking questions in a debriefing session
after test users have finished their tasks? There's no reason not to do this, but do not
expect too much. People often don't remember very much about problems they have
faced, even after a short time. Clayton remembers vividly watching a test user battle with
a text processing system for hours, and then asking afterwards what problems they had
encountered. "That wasn't too bad, I don't remember any particular problems," was the
answer. He interviewed a real user of a system who had come within one day of quitting
a good job because of failure to master a new system; they were unable to remember any
specific problem they had had. Part of what is happening appears to be that if you work
through a problem and eventually solve it, even with considerable difficulty, you
remember the solution but not the problem.
There is an analogy here to those hidden picture puzzles you see on kids' menus at
restaurant: there are pictures of three rabbits hidden in this picture, can you find them?
When you first look at the picture you can't see them. After you find them, you can not
help seeing them. In somewhat the same way, once you figure out how something works
it can be hard to see why it was ever confusing.
Something that might help you get more info out of questioning at the end of a test is
having the test session on video so you can show the test user the particular part of the
task you want to ask about. But even if you do this, don't expect too much: the user may
not have any better guess than you have about what they were doing.
Another form of debriefing that is less problematic is asking for comments on specific
features of the interface. People may offer suggestions or have reactions, positive or
negative, that might not otherwise be reflected in your data. This will work better if you
can take the user back through the various screens they've seen during the test.
4.0 CONCLUSION
In this unit, you have been introduced to evaluating designs with users. Choosing the
users to test, getting the users to know what to do, providing the necessary systems and
188