Russel-Research-Method-in-Anthropology

644 Chapter 20
data to test your theory. As replications accumulate for questions of importance
in the social sciences, the question of statistical power becomes more
and more important.
So, what’s the right amount of statistical power to shoot for? Cohen (1992)
recommends that researchers plan their work—that is, set the effect size they
recognize as important, set the level of statistical significance they want to
achieve (.05, for example, or .01), and calculate the sample size—in order to
achieve a power of .80.
A power value of .80 would be an 80% chance of recognizing that our original
hypothesis is really true and a 20% chance of rejecting our hypothesis
when it’s really true. If you shoot for a power level of much lower than .80,
says Cohen, you run too high a risk of making a Type II error.
On the other hand, power ratings much higher than .80 might require such
large n’s that researchers couldn’t afford them (ibid.:156). If you want 90%
power for a .01 (1%) two-tailed test of, say, the difference between two Pearson’s
r’s, then, you’d need 364 participants (respondents, subjects) to detect
a difference of .20 between the scores of the two groups. If you were willing
to settle for 80% power and a .05 (5%) two-tailed test, then the number of
participants drops to 192. (To find the sample size needed for any given level
of power, see Kraemer and Theimann 1987:105–112.)
The Shotgun Approach
A closely related issue concerns ‘‘shotgunning.’’ This involves constructing
a correlation matrix of all combinations of variables in a study, and then relying
on tests of significance to reach substantive conclusions.
Kunitz et al. (1981) studied the determinants of hospital utilization and surgery
in 18 communities on the Navajo Indian Reservation during the 1970s.
They measured 21 variables in each community, including 17 independent
variables (the average education of adults, the percentage of men and women
who worked full-time, the average age of men and women, the percentage of
income from welfare, the percentage of homes that had bathrooms, the percentage
of families living in traditional hogans, etc.) and four dependent variables
(the rate of hospital use and the rates for the three most common types
of surgery). Table 20.20 shows the correlation matrix of all 21 variables in this
study.
There are n(n1)/2 pairs in any list of items, so, for a symmetric matrix of
21 items there are 210 possible correlations. Kunitz et al. point out in the footnote
to their matrix that, for n 18, the 0.05 level of probability corresponds
to r 0.46 and the 0.01 level corresponds to r 0.56. By the Bonferonni
correction, they could have expected: