Local Evaluation of Children's Services Learning from the Children's ...

ConclusionsLocal evaluators have responded to the challenges of evaluating the complex, multilayeredarrangements developed by Children’s Fund partnerships by creativelyadopting evaluation methods. Many have drawn on multiple quantitative andqualitative methods, as well as data sets at a number of different levels, to show theimpacts of projects and other activities, or used methods differentially to answerquestions about different aspects of partnerships’ work. A widely experienceddilemma, however, relates to the difficulties in definitively demonstrating the impactsattributable to specific projects or activities, reflecting the complexity of the initiativeand limited time frames within which evaluations have worked. Some evaluators arevery forthright in acknowledging the limits to what they are able to show, whilst othersacknowledge the importance of openness and transparency in the evaluationprocess and the methods used in terms of data collection, analysis andinterpretation. What this highlights is the importance of evaluators managingstakeholders’ expectations about what evaluation is able to show, and its limits.Many evaluators also recognise the value of evaluation evidence based on serviceusers, and in particular, children and young people’s voices, although the extent towhich children and young people have been engaged in evaluation activities is veryvaried. The idea that children’s voices should be heard, valued and acted upon isconsistent with the participative ethos of the Children’s Fund initiative. However, anumber of factors have made it difficult for evaluators to work in this way. It is widelyrecognised that additional time and resources may be required to effectively workwith children and young people. Moreover, many evaluators describe a degree ofcircumspection, and occasionally hostility to qualitative methods such as interviewingin which children and young people express their views about their use of projectsand their perceptions of the benefits of using them. Such methods are frequentlyseen as more ‘anecdotal’, with the implication that they produce lesser forms ofevidence than ‘hard’, quantitative outcome. Nevertheless, the value of qualitativemethods in social policy research is increasingly recognised in governmentdepartments. This highlights the need for evaluators to carefully work withstakeholders to develop clear understandings of the ways difficult sets of evaluationmethods can be effectively used in combination to assess the work of differentelements of programmes’ work.Conclusions 61