Thinking and Deciding

METHODS FOR EMPIRICAL RESEARCH 37
10. The subject returns to the original goal of figuring out what goes in the lower
right, a task quickly accomplished once the subgoal of finding the rule is
achieved.
This analysis shows how the search-inference framework enables us to categorize
the moves that a thinker makes in the course of thinking. Notice that a given
move can belong to two different categories, because the move may have two different
functions; for example, in move 6, the same object is both a possibility and
a new subgoal. A given phase in an episode of thinking can contain other episodes
of thinking, which can contain others, and so on. For example, the task of searching
for the goal might involve trying to understand the instructions, which might involve
searching for possibilities and for evidence about the meaning of words such as “matrix”
(if one did not know the meaning already). As an exercise, you might find it
useful to generate another verbal protocol of your own thinking about some problem
and analyze it in this way.
Psychologists have developed a great variety of other methods for analyzing
think-aloud protocols. (Ericsson and Simon, 1980, review a number of these.) Different
approaches use different units of analysis. Some investigators allow the system
of analysis itself to define the unit: I did this in the example just given, using as units
the categories of the search-inference framework. Other investigators divide the protocol
into linguistic units, such as sentences. Others use time measurements, dividing
the protocol into 5- or 10-second units and analyzing what is happening (or not happening)
in each unit. Approaches also differ in the categories used. The method
of analysis is closely linked with the investigator’s own goals and the theoretical or
practical questions that led to the work.
Despite the extensive use of this method, many doubts have been raised over the
years about its adequacy:
1. Some mental processes do not produce much that is accessible for conscious
report. Or the processes may go by too quickly for the subject to remember
them.
2. The instruction to think aloud may induce subjects to think differently than
they ordinarily would. For example, they could think less quickly because of
the need to verbalize everything. Verbalization could interfere with thinking,
or it could help by forcing thinkers to be more careful. Both of these results
have been found, but it has also been found that in many tasks verbalization
has no apparent effect (Ericsson and Simon, 1980).
3. Verbal protocols might be misleading with respect to the underlying determinants
of the subjects’ behavior. For example, suppose that you are deciding
whether to buy a used television set, and you say, “The picture is nice, but
the sound isn’t very good, and $200 is too expensive. I’ll keep looking.” One
might infer from this that you are following a rule that you should not pay more
than $200, no matter what. Although this may be true, it may instead be true