Thinking and Deciding

182 HYPOTHESIS TESTING
Table 7.5: Test results and incidence of two diseases for 100 patients
Test result Disease A Disease B
Positive 64 8
Negative 16 12
useful test, with a much lower false-positive rate than .4, even if it also has a lower
hit rate than .8. We will not think of looking for such a test, however, if we do not
ask how likely we are to observe the same results if the alternative hypothesis is true.
The same need to consider whether a result is consistent with alternative hypotheses
— before looking for the result — occurs in science. As we can see from
the present example, this heuristic can be justified by a normative theory based on
the idea of expected utility, a theory we shall explore in Chapter 10.
Conclusion
Actively open-minded thinking can avoid one of the main biases in hypothesis testing,
the failure to consider alternative hypotheses (possibilities). In addition, more
careful search for goals can help us avoid confusing of the goal of seeking information
that is useful in decision making with other goals, such as the goal of seeking
information in general, or that of obtaining positive results. These applications of the
prescriptive theory of actively open-minded thinking can bring us closer to the normative
theory of hypothesis testing as specified by probability and expected-utility
theory.
There is more to science than hypothesis testing, however. In formulating hypotheses
to test, the scientist must not only consider whether they are testable but
also whether they are likely to be true. To meet the latter requirement, we must think
well about what we already know (and the better we know it, the better we can think
about it). We must make sure that our hypotheses are consistent with the evidence
already available before we bother to collect new data.
The rigorous standards of science are often put in opposition to the idea of “learning
from experience.” In the next chapter, we ask just how good we are at learning
from experience about the relation between one variable and another.