What Works for Children with Literacy Difficulties? - Digital ...

- reporting only raw scores, either with no control group results at all, or without the
standard deviations necessary to calculate an effect size
- internal inconsistencies which could not be resolved from the information available.
Also, Success for All could not be included because no British evaluation data were yet
available. In the USA, this programme has been well researched and shown to be highly
effective, and it is being implemented in several areas in the UK, especially Nottingham City
and Nottinghamshire.
A number of other widely used schemes were not included because, again, no evaluation data
could be found, either in the research literature or via direct approach to scheme originators or
users. A list of these schemes is given in the Acknowledgments (pp.v-vi).
The project at Saint Lawrence School was singled out for special comment in the first edition.
It was designed and implemented by the school itself, which also carried out and reported its
own quantitative evaluation – a rare and worthwhile example of teacher-led research. Two
other such initiatives, which also could not be included in this edition, deserve the same
accolade: a scheme implemented in South Gloucestershire promising ‘a flying start in
phonics’, and Underattainers Can Achieve! at Great Crosby Catholic Primary School in
Sefton.
4.2 The quality of the research
Having reviewed the Paired Reading literature, Topping and Lindsay (1992, p.201)
commented, with academic restraint: ‘The quality of studies was extremely varied.’ The
literature surveyed for the first edition of this report varied from the meticulous to the very
weak. The most meticulous was the only randomised controlled trial, the original Cumbria
Reading with Phonology Project, now called Reading Intervention, though there were several
other well-conducted experiments. The number of studies excluded from the analysis then
(for whatever reason) was considerably larger than the number retained. Most of the excluded
studies provided no quantitative data at all, and many of those which did provide such data
were unusable, either because of basic design faults (too few subjects, same test used both pre
and post over too short an interval, etc.) or because the data were unclear (averages did not
correspond with the individual scores, too many children scored zero or maximum, etc.).
The trawl for this edition was more focused and did not produce so much unusable
information. There was still some, however. The major deficiencies this time were again
inadequate sample sizes, the use of unstandardised instruments, and failure to provide data
from which an impact measure could be calculated.
This is not to say that those studies which we have included necessarily told us everything we
needed to know. The frequency of the phrase ‘not stated’ in the Appendix shows how much
information was missing, sometimes even from the reports of quite large-scale,
independently-funded evaluations.
Three particular problems arose from the tests used in the 25 studies (the tests are listed in the
Appendix). First, some of the tests were old even when used in the relevant studies.
60