Local Evaluation of Children's Services Learning from the Children's ...

More documents

Recommendations

Info

ConclusionsLocal evaluators have responded to the challenges of evaluating the complex, multilayeredarrangements developed by Children’s Fund partnerships by creativelyadopting evaluation methods. Many have drawn on multiple quantitative andqualitative methods, as well as data sets at a number of different levels, to show theimpacts of projects and other activities, or used methods differentially to answerquestions about different aspects of partnerships’ work. A widely experienceddilemma, however, relates to the difficulties in definitively demonstrating the impactsattributable to specific projects or activities, reflecting the complexity of the initiativeand limited time frames within which evaluations have worked. Some evaluators arevery forthright in acknowledging the limits to what they are able to show, whilst othersacknowledge the importance of openness and transparency in the evaluationprocess and the methods used in terms of data collection, analysis andinterpretation. What this highlights is the importance of evaluators managingstakeholders’ expectations about what evaluation is able to show, and its limits.Many evaluators also recognise the value of evaluation evidence based on serviceusers, and in particular, children and young people’s voices, although the extent towhich children and young people have been engaged in evaluation activities is veryvaried. The idea that children’s voices should be heard, valued and acted upon isconsistent with the participative ethos of the Children’s Fund initiative. However, anumber of factors have made it difficult for evaluators to work in this way. It is widelyrecognised that additional time and resources may be required to effectively workwith children and young people. Moreover, many evaluators describe a degree ofcircumspection, and occasionally hostility to qualitative methods such as interviewingin which children and young people express their views about their use of projectsand their perceptions of the benefits of using them. Such methods are frequentlyseen as more ‘anecdotal’, with the implication that they produce lesser forms ofevidence than ‘hard’, quantitative outcome. Nevertheless, the value of qualitativemethods in social policy research is increasingly recognised in governmentdepartments. This highlights the need for evaluators to carefully work withstakeholders to develop clear understandings of the ways difficult sets of evaluationmethods can be effectively used in combination to assess the work of differentelements of programmes’ work.Conclusions 61
The importance of providing findings that are relevant, useful, timely and accessibleto a range of stakeholders appears to be widely recognised by local evaluators, as isthe importance of providing realistic recommendations to partnerships. Many localevaluators regarded ongoing dialogue with partnerships as critical to successfullyinfluencing their work through engagement in partnership structures such asevaluation sub-groups or disseminating material through events, rather than simplywriting reports for strategic audiences. Indeed, running events may be a moreinclusive way of disseminating their findings to strategic stakeholders as well as frontlineservice providers and service users including children and young people.However, it is evident that many local evaluators have, in practice, experienceddifficulties influencing the development of partnerships’ strategies and practices, orare unclear about how their findings have been translated into strategies andpractice. Some evaluators suggested they had experienced difficulties maintainingtheir independence, in reporting on negative aspects of partnerships’ work or thatpartnerships had drawn on evaluation material selectively to demonstrate thesuccessful aspects of their work or to legitimise decisions. This was seen in thecontext of pressures from ongoing change nationally and within local authority areas.Partnership board members’ multiple priorities and their commitments to their ownagencies and other partnerships meant they were unable to act on evaluationfindings or had insufficient time to do so. This highlights the challenge that localevaluators have had in providing material that is relevant and useful to partnershipswhilst having the space to produce balanced, independent accounts of thepartnerships’ work.62Conclusions
Page 1 and 2:
RESEARCHLocal Evaluation ofChildren
Page 4 and 5:
When citing this report please show
Page 6 and 7:
ContentsExecutive SummaryIIntroduct
Page 8 and 9:
Local Evaluation of the Children’
Page 10 and 11:
10. Local evaluators highlighted th
Page 12 and 13:
20. Some evaluators involved childr
Page 14 and 15:
successes of partnerships’ work (
Page 16:
38. Conditions reinforcing effectiv
Page 19 and 20:
Children’s Fund initiative, namel
Page 21 and 22:
engaging with policy makers, practi
Page 23 and 24:
starting points and timetables of d
Page 25 and 26:
1. Process evaluation: critical ass
Page 27 and 28: time commitments and pressures amon
Page 29 and 30: tantamount to auditing when origina
Page 31 and 32: to a certain extent, because I was
Page 33 and 34: acknowledge that findings presented
Page 35 and 36: methods that were seen as being app
Page 38 and 39: Chapter 2: Methods Adopted by Child
Page 40 and 41: place. In one Wave One evaluation t
Page 42 and 43: provided by projects weakened the a
Page 44 and 45: confidently attribute any trends in
Page 46 and 47: A Wave Two evaluation report descri
Page 48 and 49: • Recruitment of the researcher:
Page 50 and 51: drawn on by evaluators include semi
Page 52 and 53: Chapter 3: How Local Evaluation Sup
Page 54 and 55: participation and work towards the
Page 56 and 57: highlighted in Table 1). These incl
Page 58 and 59: of had an impact. So we used a case
Page 60 and 61: Similarly an evaluator of a Wave Tw
Page 62 and 63: Evaluators sometimes struggled to a
Page 64: 6. A number of factors inhibit eval
Page 67 and 68: A number of evaluations described h
Page 69 and 70: 4.2 Key Messages from Local Evaluat
Page 71 and 72: for staff, typically dedicated part
Page 73 and 74: Where partnership working was descr
Page 75 and 76: strategic changes in children’s s
Page 77: 9. Evaluators reported that some co
Page 81 and 82: National Evaluation of the Children
show all

Local Evaluation of Children's Services Learning from the Children's ...

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?