What Works for Children with Literacy Difficulties? - Digital ...

5) The units in which average scores and s.d’s are stated are almost always either
reading/spelling ages or standardised scores, very rarely both. Raw scores were used in a
few cases, e.g. Somerset (4).
6) Where the units of measurement are r.a’s/s.a’s, gain is given in months of r.a./s.a.
7) Ratio gain (RG): This is defined by Topping and Lindsay (1992, p.201) as ‘the gain in
Reading age made by a subject on a reading test during a chronological time span,
expressed as a ratio of that time span; that is, ratio gain equals reading age gain in months
divided by chronological time in months’. For a group, this can be stated as the formula
(average reading age in months at post-test) - (average reading age in months at pre-test)
_____________________________________________________________________
time elapsed in months
(The definition and formula are obviously applicable to spelling too.)
Thus calculating an RG does not require data from a control group - but where any nonexperimental
group is present its RG can and should be calculated too; and the RGs for
non-experimental groups are valid for those groups independently of whether they were
properly equivalent to the experimental group, because each RG is calculated for the
relevant group of subjects without reference to any other group.
The dispersal of scores (as shown in the standard deviation) is ignored in RGs – only the
average reading ages at pre- and post-test and time elapsed are used. RG is therefore a
statistically unsophisticated device; but, as Topping and Lindsay further point out, using
raw gains instead ‘renders the highly heterogeneous literature very difficult to
summarise’. Also, since over half of the evaluations surveyed here used reading ages as
their reporting units (see below) it seemed appropriate to use RGs in attempting to make
comparisons between those interventions.
However, RGs do take account of the length of time over which an intervention achieves
its impact - as shown in the formula, this is done by dividing the gain in months of reading
age (r.a.) by the number of months between pre- and post-test.
RGs were explicitly stated in only a few reports. However, the r.a. data required for
calculating RGs were given in many more. Some reports did not use tests which yielded
r.a’s/s.a’s, and therefore RGs could not be calculated for them.
8) Effect size: This is a more statistically based metric. It involves dividing the difference
between the gain made by the experimental group and that made by the control group by a
relevant standard deviation, and the result is expressed as a decimal of an s.d. Therefore
the top line of the relevant formula can be stated as:
(average gain of experimental group) - (average gain of control group)
This part of the formula can be applied equally to r.a’s, s.a’s, standardised scores and raw
scores. The problems arise with the choice of control groups and of the appropriate s.d. to
use as the divisor.
74