RESEARCH METHOD COHEN ok

EFFECT SIZE 521
differential measure of effect size is more useful
than the blunt edge of statistical significance.
An effect size is
simply a way of quantifying the difference between
two groups. For example, if one group has had an
‘experimental treatment’ and the other has not (the
‘control’), then the Effect Size is a measure of the
effectiveness of the treatment.
(Coe 2000: 1)
It tells the reader ‘how big the effect is, something
that the p value [statistical significance] does not
do’ (Wright 2003: 125). An effect size (Thompson
2002: 25) ‘characterizes the degree to which
sample results diverge from the null hypothesis’; it
operates through the use of standard deviations.
Wood (1995: 393) suggests that effect size can
be calculated by dividing the significance level
by the sample size. Glass et al. (1981: 29, 102)
calculate the effect size as:
(mean of experimental group – mean of control group)
standard deviation of the control group
Coe (2000: 7), while acknowledging that there
is a debate on whether to use the standard
deviation of the experimental or control group
as the denominator, suggests that that of the
control group is preferable as it provides ‘the best
estimate of standard deviation, since it consists
of a representative group of the population who
have not been affected by the experimental
intervention’. However, Coe (2000) also suggests
that it is perhaps preferable to use a ‘pooled’
estimate of standard deviation, as this is more
accurate than that provided by the control group
alone. To calculate the pooled deviation he
suggests that the formula should be:
√
(N E − 1)SD 2 E
SD pooled =
+ (N C − 1)SD 2 C
N E + N C − 2
where N E = number in the experimental group,
N C = number in the control group, SD E =
standard deviation of the experimental group and
SD C = standard deviation of the control group.
The formula for the pooled deviation then
becomes (Muijs 2004: 136):
(mean of experimental group – mean of control group)
pooled standard deviation
where the pooled standard deviation = (standard
deviation of group 1 + standard deviation of
group 2).
There are several different calculations of
effect size, for example (Richardson 1996;
Capraro and Capraro 2002: 771): r 2 , adjusted
R 2 , η 2 , ω 2 , Cramer’s V, Kendall’s W, Cohen’s
d, and Eta. Different kinds of statistical
treatments use different effect size calculations.
For example, the formula given by Muijs (2004)
here yields the statistic termed Cohen’s d.
Further details of this, together with a facility
which calculates it automatically, can be
found at http://www.uccs.edu/∼lbecker/psy590/
escalc3.htm.
An effect size can lie between 0 to 1 (some
formulae yield an effect size that is larger than
1–seeCoe2000).InusingCohen’sd:
0–0.20 = weak effect
0.21–0.50 = modest effect
0.51–1.00 = moderate effect
>1.00 = strong effect
In correlational data the coefficient of correlation
is used as the effect size in conjunction with
details of the direction of the association (i.e. a
positive or negative correlation). The coefficient
of correlation (effect size) is interpreted thus:
< 0 + / − 0.1 weak
< 0 + / − 0.3 modest
< 0 + / − 0.5 moderate
< 0 + / − 0.8 strong
≥+/− 0.8 verystrong
We provide more detail on interpreting correlation
coefficients later in this chapter. However,
Thompson (2001; 2002) argues forcibly
against simplistic interpretations of effect size as
‘small’, ‘medium’ and ‘large’, as to do this commits
the same folly of fixed benchmarks as that
of statistical significance. He writes that ‘if people
Chapter 24