Statistics for the Behavioral Sciences by Frederick J. Gravetter, Larry B. Wallnau ISBN 10: 1305504917 ISBN 13: 9781305504912

SECTION 8.5 | Concerns about Hypothesis Testing: Measuring Effect Size 251
and the z-score for M = 51 is
z 5 M 2m 51 2 50
5 5 1 s M
2 2 5 0.50
This z-score fails to reach the critical boundary of z = 1.96, so we fail to reject the null
hypothesis. In this case, the 1-point difference between M and μ is not significant because
it is being evaluated relative to a standard error of 2 points.
Now consider the outcome with a sample of n = 400. With a larger sample, the standard
error is
and the z-score for M = 51 is
s M
5 s Ïn 5 10
Ï400 5 10
20 5 0.50
z 5 M 2m 51 2 50
5 5 1
s M
0.5 0.5 5 2.00
Now the z-score is beyond the 1.96 boundary, so we reject the null hypothesis and conclude
that there is a significant effect. In this case, the 1-point difference between M and μ is
considered statistically significant because it is being evaluated relative to a standard error
of only 0.5 points.
■
The point of Example 8.5 is that a small treatment effect can still be statistically significant.
If the sample size is large enough, any treatment effect, no matter how small, can be
enough for us to reject the null hypothesis.
■ Measuring Effect Size
As noted in the previous section, one concern with hypothesis testing is that a hypothesis
test does not really evaluate the absolute size of a treatment effect. To correct this problem,
it is recommended that whenever researchers report a statistically significant effect, they
also provide a report of the effect size (see the guidelines presented by L. Wilkinson and the
APA Task Force on Statistical Inference, 1999). Therefore, as we present different hypothesis
tests we will also present different options for measuring and reporting effect size. The
goal is to measure and describe the absolute size of the treatment effect in a way that is not
influenced by the number of scores in the sample.
DEFINITION
A measure of effect size is intended to provide a measurement of the absolute
magnitude of a treatment effect, independent of the size of the sample(s) being used.
One of the simplest and most direct methods for measuring effect size is Cohen’s d.
Cohen (1988) recommended that effect size can be standardized by measuring the mean
difference in terms of the standard deviation. The resulting measure of effect size is computed
as
Cohen’s d 5
mean difference
standard deviation 5 m 2m treatment no treatment
s
(8.1)
For the z-score hypothesis test, the mean difference is determined by the difference
between the population mean before treatment and the population mean after treatment.
However, the population mean after treatment is unknown. Therefore, we must use the
mean for the treated sample in its place. Remember, the sample mean is expected to be