Thinking and Deciding

368 QUANTITATIVE JUDGMENT
judgments of the exact same stimulus configurations are not identical.
He is subject to all those human frailties which lower the reliability of
his judgments below unity. And if the judge’s reliability is less than
unity, there must be error in his judgments — error which can serve no
other purpose but to attenuate his accuracy. If we could remove some
of this human unreliability by eliminating the random error in his judgments,
we should thereby increase the validity of the resulting predictions.
We can, however, think of reasons why the judge might still be better than the
model of himself. In particular, there might be interactions or curvilinear relations
(nonlinearities) that are not represented in the linear models we have been considering.
It might really be true that SATs are more important in the middle range than
the higher range, for example. If this were true, the judge would be able to take it
into account, and the linear model of the judge would not.
The judge might also have additional information not included in the model, such
as observations made during an interview with the applicant. This gives the judge an
unfair advantage over the model, however, since it is possible for the judge to assign
numerical ratings to that information and include these in the model.
In the great variety of situations examined so far, interactions and nonlinearities
have never been as important as the error to which Goldberg refers. There are
sometimes nonlinearities and interactions in the true situation, and there are sometimes
nonlinearities and interactions in the judge’s judgments, but the nonlinearities
and interactions expressed in his actual judgments have little to do with the nonlinearities
and interactions actually present (Camerer, 1981). (Later in this chapter,
however, we shall consider a case in which subjects make use of interactions among
features when deciding whether a stimulus is a member of a category.)
In general, then, it is better to use the formula than the judge, because the formula
eliminates the error. This method is called bootstrapping. We pull ourselves up by
our bootstraps. We improve our judgment by modeling ourselves. A good example
of the successful use of bootstrapping methods is a formula devised by Gustafson,
Tianen, and Greist (1981), which predicts suicide attempts in psychiatric patients on
the basis of variables that can be assessed in clinical interviews (such as extent of
suicide plans and degree of isolation from other people). The predictions are more
accurate than those made holistically by the judges.
When we use bootstrapping, we estimate weights from holistic decisions, rather
than from systematic considerations of tradeoffs; we calculate the weights so as to
best explain a person’s holistic ratings. 3 Although bootstrapping does better than
holistic judgments, it is still derived from such judgments. We noted in Chapter 12
that less important dimensions are underweighed in holistic judgments. Bootstrapping
solves the problem of random error, but it does not remove the effects of this underweighing.
In a bootstrapping procedure the dimensions considered less important
in fact receive less weight, relative to other dimensions, than in a MAUT procedure
3 We also assume that the utility scales are linear.