Local Evaluation of Children's Services Learning from the Children's ...

More documents

Recommendations

Info

IntroductionAims, scope and methodsThere is a growing academic and policy literature on developments in thinking abouthow evaluation contributes to evidence based policy and practice. Traditional‘hierarchies’ of research and evaluation methods that endorse experimental andquantitative methods are being set aside in favour of combined quantitative andqualitative methods for evaluating complex social policy initiatives. The importance ofembracing the experiences, perspectives and accounts of all stakeholders includingservice users and children and young people, and making evaluation findingsaccessible to multiple audiences, not just professionals and academics, have beenimportant developments in thinking on evaluation. There is also a growingunderstanding of the complex ways evaluation may (or may not) be used by policymakers and practitioners.This report considers the local evaluation of the Children’s Fund, a national initiativethat fully embraces the importance of evaluation in supporting partnerships to assessand develop their programmes of work. The nature and complexity of the initiativedoes, however, present a number of methodological and practical challenges toevaluators. One challenge is the highly divergent strategies and practices required atthe local level, to tackle what are complex, multi-layered and long-term issues ofsocial exclusion. This makes it particularly difficult to assess the effectiveness of theinitiative using traditional evaluation methods (Edwards & Fox, 2005). Suchchallenges have resulted in pragmatic creativity in the ways local evaluations havesought to demonstrate outcomes and support Children’s Fund partnerships’ planningin a context of rapid national and local policy change. The aim of this report is toreview the range of approaches adopted by local evaluators as well asacknowledging some of the tensions and compromises that have characterised theirwork. The report focuses on exploring four major themes:• The roles, orientations and principles of local evaluators;• The evaluation methods that have been adopted by local evaluations;• The ways local evaluations have sought to support partnerships’ decisionmaking;• Key messages emerging from local evaluators’ reports and the ways localevaluators have sought to understand the underpinning themes of theIntroduction 1
Children’s Fund initiative, namely, prevention, participation and partnershipworking.This report is based on the analysis of local external evaluation reports that havebeen made available to the National Evaluation of the Children’s Fund (NECF) todate 1 and the analysis of detailed, semi-structured interviews conducted withselected local evaluators (n=10). Interviewees were drawn from NECF case studypartnerships together with a sample of members of the NECF Local Evaluators’Reference Group. The programme managers’ questionnaire survey conducted by theNECF was also drawn on to highlight some of the ways programme managers sawlocal evaluation being used. 2 The emphasis of the analysis presented in this report isto illustrate the diversity of approaches and methods employed by local evaluators 3rather than exhaustively catalogue all local evaluation activity in all 149 partnerships.Hence, it is not possible for this report to quantify the numbers of evaluators adoptingdifferent approaches and methods. Neither is it the intention of this report to assessthe quality of individual evaluations or particular evaluation reports. To this endagencies/organisations conducting evaluations and individual Children’s Fundpartnerships are not identified to protect their anonymity.Developments in evaluation and evidence based policyIn the UK there is growing emphasis on the use of evidence in policy makingreflecting the current government’s pragmatic ideology of ‘what works’ by whichdecisions at national and local levels are expected to be founded on robust evidence.This has led to considerable expansion in evaluation infrastructure across the country(Solesbury, 2001). Nevertheless, it is recognised that key social problems such associal exclusion and health inequalities have multiple and interrelated facets andcauses including poverty, unemployment, family background, education,neighbourhood, opportunity and lifestyle. Hence, there can be no simple, causalassociation between a policy intervention and outcomes (Percy-Smith, 2000;Pierson, 2002; Coote, et al., 2004). Such recognition is reflected in complex, multiplelayeredpolicy responses to tackling social exclusion and the acknowledgement thata number of agencies would need to work in partnership to address these issues.1 At the time of writing reports relating to 80 partnerships were available for analysis.2 Based on 120 Children’s Fund programme managers’ responses (October-November,2005).3 The local evaluation reports represent the three Children’s Fund waves, regional spread,type of local authority (rural, urban, unitary, two-tier, metropolitan authorities) and agencyconducting the evaluation (university, consultant/consultancy, voluntary sector organisation).2Introduction
Page 1 and 2: RESEARCHLocal Evaluation ofChildren
Page 4 and 5: When citing this report please show
Page 6 and 7: ContentsExecutive SummaryIIntroduct
Page 8 and 9: Local Evaluation of the Children’
Page 10 and 11: 10. Local evaluators highlighted th
Page 12 and 13: 20. Some evaluators involved childr
Page 14 and 15: successes of partnerships’ work (
Page 16: 38. Conditions reinforcing effectiv
Page 21 and 22: engaging with policy makers, practi
Page 23 and 24: starting points and timetables of d
Page 25 and 26: 1. Process evaluation: critical ass
Page 27 and 28: time commitments and pressures amon
Page 29 and 30: tantamount to auditing when origina
Page 31 and 32: to a certain extent, because I was
Page 33 and 34: acknowledge that findings presented
Page 35 and 36: methods that were seen as being app
Page 38 and 39: Chapter 2: Methods Adopted by Child
Page 40 and 41: place. In one Wave One evaluation t
Page 42 and 43: provided by projects weakened the a
Page 44 and 45: confidently attribute any trends in
Page 46 and 47: A Wave Two evaluation report descri
Page 48 and 49: • Recruitment of the researcher:
Page 50 and 51: drawn on by evaluators include semi
Page 52 and 53: Chapter 3: How Local Evaluation Sup
Page 54 and 55: participation and work towards the
Page 56 and 57: highlighted in Table 1). These incl
Page 58 and 59: of had an impact. So we used a case
Page 60 and 61: Similarly an evaluator of a Wave Tw
Page 62 and 63: Evaluators sometimes struggled to a
Page 64: 6. A number of factors inhibit eval
Page 67 and 68:
A number of evaluations described h
Page 69 and 70:
4.2 Key Messages from Local Evaluat
Page 71 and 72:
for staff, typically dedicated part
Page 73 and 74:
Where partnership working was descr
Page 75 and 76:
strategic changes in children’s s
Page 77 and 78:
9. Evaluators reported that some co
Page 79 and 80:
The importance of providing finding
Page 81 and 82:
National Evaluation of the Children
show all

Local Evaluation of Children's Services Learning from the Children's ...

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?