Thinking and Deciding

THE PSYCHOLOGY OF HYPOTHESIS TESTING 181
Table 7.4: Joint probabilities, p(H & D) =p(D|H) · p(H), for water example
Question Answer H ∼H
1. jug in refrigerator yes .085 .12
no .765 .03
2. refrigerator old yes .595 .06
no .255 .09
hypothesis is true. The hypothesis H is that the refrigerator stopped working and the
ice melted. Consider two possible data, D1, a yes answer to “Had there been a large
jug of water in the refrigerator?”, and D2, a yes answer to “Was the refrigerator
old?” Assume that p(H) =.85 (so that p(H2) = .15), that p(D1|H) =.1, p(D1|∼
H) =.8, p(D2|H) =.7, and p(D2|∼H) =.4. Using the multiplication rule to
multiply each conditional probability by the prior probability of the corresponding
hypothesis, we get the results in Table 7.4. (Bear in mind that “no” is the opposite
of “yes,” so the sum of these two joint probabilities must equal the probability of the
hypothesis.) Without asking either question, the probability of guessing correctly
is .85, the probability of H. If we ask question 1, we could guess correctly with
probability .885 (.765 + .12) after asking the question, but question 2 provides no
benefit at all, since we would guess that H is true no matter what the answer to the
question.
Question 1 will this have a greater beneficial effect on the probability of guessing
correctly, and we can determine this — given our probabilities — before we even
ask the question. We can list the possible answers to a given question or the possible
results of a test, and we can ask how likely each result is, given each hypothesis, and
how much it would improve our chance of guessing correctly.
Utility and alternative hypotheses
We can use this approach to understand the value of considering alternative hypotheses,
as discussed earlier in this chapter. Suppose that we think that the probability
that a patient has disease A is .8. We know of a test that yields a positive result in
80% of the patients with disease A. Is the test worth doing?
Many would say yes. I suggest that we would need to know more before we
decide. A good heuristic is to ask what other disease the patient may have if she does
not have disease A. Suppose the answer is disease B. Then we would want to know
what the probability of a positive result from that same test would be if the patient
had disease B. Suppose the probability is .4. Then, for 100 imaginary patients, we
would have a table like Table 7.5. Even if the test is negative, disease A will be our
best guess. Therefore, the test has 0 utility. The problem is that the test has too high
a “false-positive” rate (8 out of 20 patients, or .4). Perhaps we could find a more