Statistics for the Behavioral Sciences by Frederick J. Gravetter, Larry B. Wallnau ISBN 10: 1305504917 ISBN 13: 9781305504912

SECTION 8.5 | Concerns about Hypothesis Testing: Measuring Effect Size 253
Now consider the treatment effect shown in Figure 8.9(b). This time, the treatment produces
a 15-point mean difference in IQ scores; before treatment the average IQ is 100, and
after treatment the average is 115. Because IQ scores have a standard deviation of σ = 15,
the 15-point mean difference now appears to be large. For this example, Cohen’s d is
mean difference
Cohen’s d 5
standard deviation 5 15
15 5 1.00
Notice that Cohen’s d measures the size of the treatment effect in terms of the standard
deviation. For example, a value of d = 0.50 indicates that the treatment changed the mean
by half of a standard deviation; similarly, a value of d = 1.00 indicates that the size of the
treatment effect is equal to one whole standard deviation. (Box 8.2.)
BOX 8.2 Overlapping Distributions
Figure 8.9(b) shows the results of a treatment with
a Cohen’s d of 1.00; that is, the effect of the treatment
is to increase the mean by one full standard
deviation. According to the guidelines in Table 8.2,
a value of d = 1.00 is considered a large treatment
effect. However, looking at the figure, you may get
the impression that there really isn’t that much difference
between the distribution before treatment and
the distribution after treatment. In particular, there is
substantial overlap between the two distributions, so
that many of the individuals who receive the treatment
are not any different from the individuals who
do not receive the treatment.
The overlap between distributions is a basic fact
of life in most research situations; it is extremely
rare for the scores after treatment to be completely
different (no overlap) from the scores before treatment.
Consider, for example, children’s heights at
different ages. Everyone knows that 8-year-old children
are taller than 6-year-old children; on average,
the difference is 3 or 4 inches. However, this does
not mean that all 8-year-old children are taller than
all 6-year-old children. In fact, there is considerable
overlap between the two distributions, so that the
tallest among the 6-year-old children are actually
taller than most 8-year-old children. In fact, the
height distributions for the two age groups would
look a lot like the two distributions in Figure 8.9(b).
Although there is a clear mean difference between
the two distributions, there still can be substantial
overlap.
Cohen’s d measures the degree of separation
between two distributions, and a separation of one
standard deviation (d = 1.00) represents a large
difference. Eight-year-old children really are bigger
than 6-year-old children.
Cohen (1988) also suggested criteria for evaluating the size of a treatment effect as
shown in Table 8.2.
TABLE 8.2
Evaluating effect size with
Cohen’s d.
Magnitude of d
d = 0.2
d = 0.5
d = 0.8
Evaluation of Effect Size
Small effect (mean difference around 0.2 standard deviation)
Medium effect (mean difference around 0.5 standard deviation)
Large effect (mean difference around 0.8 standard deviation)
As one final demonstration of Cohen’s d, consider the two hypothesis tests in Example 8.5.
For each test, the original population had a mean of μ = 50 with a standard deviation of
σ = 10. For each test, the mean for the treated sample was M = 51. Although one test
used a sample of n = 25 and the other test used a sample of n = 400, the sample size is
not considered when computing Cohen’s d. Therefore, both of the hypothesis tests would
produce the same value:
Cohen’s d 5
mean difference
standard deviation 5 1 10 5 0.10