Action Research A Methodology for Change and Development

ACTION RESEARCH IN THE EVALUATION OF A NATIONAL PROGRAMME 155
Designing a ‘supportive evaluation’ of the ICT Test
Bed project
The overall aims of the ICT Test Bed project are focused on whole school
improvement (limited in the colleges to three curriculum areas) across five
themes: teaching and learning, leadership and management, workforce
development, collaboration between the cluster schools and colleges and
improved links with pupils’ homes and the community. The three clusters,
comprising in all 28 schools and three further education colleges, are located
in: an inner suburb of a West Midlands conurbation with a large number of
ethnic minority and asylum seeker families; a rural district in a former coalmining
area; and a suburb in the east end of London. All are areas of high
social deprivation and the aim is to see if very high levels of ICT equipment
can make a significant difference to the educational opportunities of children
in the area and raise their levels of achievement. Overall project
funding in the first year was for £20 million, rising to a total of £34 million
over a four-year period. The very large scale of this funding is significant
because of the high expectations it raises in the minds of sponsors.
Although the schools and colleges were specifically chosen because of the
extreme difficulties they faced in achieving high levels of attainment for
their pupils, staff fear that the scale of the funding will be expected to
produce miracles. Improvements in national test scores and examination
results is the expressed aim. These are outcomes that would transform the
life chances of the children concerned, but, although they include a
measure of ‘value added’ in relation to socio-economic data, they are crude
measures that may not capture fully the different kinds of learning resulting
from a technology-rich education.
Supportive evaluation requires a careful balance between adopting an
independent stance in order to provide fair and honest judgements and
sympathetic engagement with the complexities of the ICT initiative. It
involves bringing knowledge from previous evaluations of similar initiatives
to assist project participants in overcoming problems and to inform
sponsors of the practical difficulties and systemic barriers that are likely to
make their initial expectations somewhat unrealistic in practice. This strong
emphasis on formative feed-back and knowledge transfer draws evaluators
closer to the project participants and has the potential to endanger their
independence. However, the alternative of withholding knowledge and
watching an initiative fail – or even more likely, watching the project director
and participants being blamed for ‘failures’ beyond their control – is
clearly unethical. Evaluators also need to resist the sponsors’ instinctive
desire to reduce complexity and their tendency to rely too much on quantitative
measures of outcomes such as test scores without the benefit of
theoretical understandings, developed from qualitative data capable of
contextualizing and illuminating the meaning of those outcomes. The