Local Evaluation of Children's Services Learning from the Children's ...

More documents

Recommendations

Info

10. Local evaluators highlighted their experience of a number of engagement andmethodological problems. Some evaluators indicated that time commitmentsand pressures among project staff and strategic stakeholders influenced theirabilities to participate in the evaluation.11. One methodological issue stems from the roles many local evaluators have asboth generating learning to develop more effective practices, and adopting arole regarded by some stakeholders as analogous to auditing Children’s Fundprojects. Indeed, some local evaluators encountered service providers whowere uncertain about the purpose of evaluation and considerable time wasspent convincing them of its benefits. Some concerns were raised that serviceproviders only offered positive accounts of their progress rather than revealingproblematic issues since they connected evaluation to decisions about futurecommissioning, a concern that was sometimes justified.12. Local evaluation is undertaken in a complex, changing policy environment.Uncertainties relating to the evolution of local level structures such aschildren’s trusts were described by a number of evaluators as problematic, aswas the announcement of budget cuts for the Children’s Fund initiative whichfor some undermined a systematic approach to conducting local evaluation.13. Local evaluators point to a number of principles and standards that haveguided them. These are: to provide useful and relevant material forpartnerships; to be responsive to change; to offer independent, balancedaccounts of partnerships’ activities; to be inclusive to different groups ofstakeholders; to use credible methods and evidence to support evaluationfindings; to adopt ethical approaches to evaluation.Methods adopted by Children’s Fund local evaluations14. Local evaluators drew on multiple quantitative and qualitative data sourcesand analysis methods that reflected the different elements of partnerships’programmes they aimed to examine. Quantitative methods, sometimescomplemented by qualitative methods, were frequently adopted for assessingthe impacts of interventions. Qualitative methods, sometimes supplementedby quantitative methods, were primarily employed for generating learningExecutive Summaryiii
about processes such as the effectiveness of partnership working,commissioning and approaches to mainstreaming.15. Quantitative methods adopted by evaluators include structured questionnairesand interviews with stakeholders and analysis of secondary statistical datasources, including the analysis of project monitoring data. Qualitative methodsdrawn on by evaluators include semi-structured or unstructured interviewswith stakeholders, focus groups, observations of project activities and analysisof partnership documents.16. Evaluators have aimed to measure the impacts of Children’s Fund projectsand other interventions on aspects of children’s lives corresponding to theChildren’s Fund sub-objectives and the Every Child Matters outcomes. Someevaluators also explored impacts in terms of how effectively projects targetedtheir services at user groups or as an assessment of the value for money ofparticular projects and other activities.17. There are a number of challenges to measuring impacts attributable tospecific Children’s Fund programmes and projects. The problems of socialexclusion tackled by the Children’s Fund are complex, multi-layered andinteracting, making it difficult to demonstrate causal links between a singleintervention and a measurable outcome. A related difficulty is the limitedextent to which time-limited local evaluation is able to identify longer-termoutcomes for children using Children’s Fund services.18. Some evaluators had difficulties responding to the Children’s Fund subobjectivesand Every Child Matters outcomes frameworks which defineoutcomes in relatively broad terms that are potentially open to a range ofinterpretations. Established statistical data sets tend not to be available thatclearly show change, although those relating to educational attainment andattendance are widely available. Some evaluators also noted the problems ofdrawing on inaccurate or inconsistent monitoring data provided by projects.19. Such challenges have led to some evaluators embracing the use of multiplemethods and multiple data sources creatively in ways that enable them tosuggest with some confidence that Children’s Fund projects have led topositive outcomes for children using the services.ivExecutive Summary
Page 1 and 2: RESEARCHLocal Evaluation ofChildren
Page 4 and 5: When citing this report please show
Page 6 and 7: ContentsExecutive SummaryIIntroduct
Page 8 and 9: Local Evaluation of the Children’
Page 12 and 13: 20. Some evaluators involved childr
Page 14 and 15: successes of partnerships’ work (
Page 16: 38. Conditions reinforcing effectiv
Page 19 and 20: Children’s Fund initiative, namel
Page 21 and 22: engaging with policy makers, practi
Page 23 and 24: starting points and timetables of d
Page 25 and 26: 1. Process evaluation: critical ass
Page 27 and 28: time commitments and pressures amon
Page 29 and 30: tantamount to auditing when origina
Page 31 and 32: to a certain extent, because I was
Page 33 and 34: acknowledge that findings presented
Page 35 and 36: methods that were seen as being app
Page 38 and 39: Chapter 2: Methods Adopted by Child
Page 40 and 41: place. In one Wave One evaluation t
Page 42 and 43: provided by projects weakened the a
Page 44 and 45: confidently attribute any trends in
Page 46 and 47: A Wave Two evaluation report descri
Page 48 and 49: • Recruitment of the researcher:
Page 50 and 51: drawn on by evaluators include semi
Page 52 and 53: Chapter 3: How Local Evaluation Sup
Page 54 and 55: participation and work towards the
Page 56 and 57: highlighted in Table 1). These incl
Page 58 and 59: of had an impact. So we used a case
Page 60 and 61:
Similarly an evaluator of a Wave Tw
Page 62 and 63:
Evaluators sometimes struggled to a
Page 64:
6. A number of factors inhibit eval
Page 67 and 68:
A number of evaluations described h
Page 69 and 70:
4.2 Key Messages from Local Evaluat
Page 71 and 72:
for staff, typically dedicated part
Page 73 and 74:
Where partnership working was descr
Page 75 and 76:
strategic changes in children’s s
Page 77 and 78:
9. Evaluators reported that some co
Page 79 and 80:
The importance of providing finding
Page 81 and 82:
National Evaluation of the Children
show all

Local Evaluation of Children's Services Learning from the Children's ...

Create successful ePaper yourself

Delete template?

Save as template?