RESEARCH METHOD COHEN ok

522 QUANTITATIVE DATA ANALYSIS
interpret effect sizes with the same rigidity that
α = .05 has been used in statistical testing, we
would merely be being stupid in another metric’
(Thompson 2001: 82–3). Rather, he avers, it is
important to avoid fixed benchmarks (i.e. cut-off
points), and relate the effect sizes found to those
of prior studies, confidence intervals and power
analyses. Wright (2003: 125) also suggests that it
is important to report the units of measurement of
the effect size, for example in the units of measure
of the original variables as well in standardized
units (e.g. standard deviations), the latter being
useful if different scales of measures are being used
for the different variables.
We discussed confidence intervals in Chapter 4.
It is the amount of the ‘true population value
of the parameter’ (Wright 2003: 126), e.g. 90
per cent of the population, 95 per cent of the
population, 99 per cent of the population. A
confidence interval is reported as 1 − α, i.e.the
level of likelihood that a score falls within a
pre-specified range of scores (e.g. 95 per cent,
99 per cent likelihood). Software for calculating
confidence intervals for many measures can be
found at http://glass.ed.asu/stats/analysis/.
The power of a test is ‘an estimate of the ability
of the test to separate the effect size from random
variation’ (Gorard 2001: 14), the ‘probability of
rejecting a specific effect size for a specific sample
size at a particular α level (i.e. the critical level
to reject H 0 )(Wright2003:126).Wright(2003)
suggests that it should be a minimum of 80 per
cent and to be typically with an α level at 5 per
cent. Software for calculating power analysis can
be found at http://www.psycho.uni duesseldorf.
de/aap/projects/gpower.
In calculating the effect size (Eta squared) for
independent samples in a t-test (discussed later)
the following formula can be used.
Eta squared =
t 2
t 2 + (N 1 + N 2 −2)
Here t = the t-value (calculated by SPSS);
N 1 = the number in the sample of group one
and N 2 = the number in the sample of group
2. Let us take an example of the results of
an evaluation item in which the two groups
are leaders/senior management team (SMT) of
schools, and teachers, (Boxes 24.16 and 24.17).
Here the t-value is 1.923, N 1 is 347 and N 2 is
653. Hence the formula is:
t 2
t 2 + (N 1 + N 2 − 2) = 1.923 2
1.923 2 + (347 + 653 − 2)
=
3.698
3.698 + 998 = 0.0037
The guidance here from Cohen (1988) is that
0.01 = a very small effect; 0.06 = a moderate
effect; and 0.14 = a very large effect. Here the
result of 0.003 is a tiny effect, i.e. only 0.3 per cent
of the variance in the variable ‘How well learners
are cared for, guided and supported’ is explained by
whether one is a leader/SMT member or a teacher.
For a paired sample t-test (discussed later)
the effect size (Eta squared) is calculated by the
following formula:
Eta squared =
t 2
t 2 + (N 1 − 1)
Let us imagine that the same group of students had
scored marks out of 100 in ‘Maths’ and ‘Science’
(Boxes 24.18 and 24.19).
Box 24.16
Mean and standard deviation in an effect size
Group statistics
How well learners are cared
for, guided and supported
Who are you N Mean SD SE mean
Leader/SMT member
Teachers
347
653
8.37
8.07
2.085
2.462
0.112
0.096