Local Evaluation of Children's Services Learning from the Children's ...

More documents

Recommendations

Info

These shifts in thinking are widely embraced at local level by area-based initiativesincluding the Children’ Fund.This recognition has also led to changes in thinking about evaluation. In the UKclinical medicine has historically embraced the need for evidence based decisionmaking more than many other sectors, although it has tended to adopt a traditionalapproach to demonstrating ‘what works’. Standards of evidence in clinical medicineemphasise experimental methods; ‘randomised control trials’ and systematic reviewsof randomised control trials in particular are seen as the ‘gold standard’ of evidence(Becker & Bryman, 2004). Conversely, qualitative methods such as interviewingprofessionals or service users would be considered the least robust forms ofevidence. Whilst ‘lay’ perspectives such as the views of service users have nothistorically been valued, such experiences are now more readily acknowledged aslegitimate and important forms of evidence (Coote, et al., 2004; Becker & Bryman,2004). Traditional experimental approaches are now widely seen as inappropriate foridentifying what works in complex community and area-based initiatives in whichthere are no simple cause-and-effect relationships between interventions andoutcomes. A Cabinet Office report, establishing a framework for assessing qualitativeevidence, clearly signals that government departments are now valuing thecontribution of qualitative methods in social policy research and evaluation (Spencer,et al. 2004). Furthermore, it is widely recognised that a combination of evaluationmethods should be used appropriately to address different questions in differentsettings (Coote, et al., 2004; Becker & Bryman, 2004).There are also changes in the ways evaluators are seen as engaging with policymakers and practitioners. Rather than being equated with traditional auditing andperformance management roles, evaluation is frequently expected to provideformative or even dialogic feedback to help policy-makers and practitioners developprogrammes of activities. This is as well as generating more sophisticated,theoretically driven understandings of concepts such as social exclusion. Someevaluation is now committed to redressing social power imbalances and embracingdiverse and sometimes contradictory experiences, perspectives and accounts ofsocial problems and policy solutions (Donaldson & Scriven, 2003). Some of thesepotential roles and orientations identified in the literature are summarised in TableOne. Whilst some of these roles are well established in evaluation practice, othersare starting to emerge as important departures in the ways evaluation is seen asIntroduction 3
engaging with policy makers, practitioners and indeed other stakeholders of socialpolicies such as communities.It is also recognised in the literature that the extent to which research and evaluationinfluence policy depends on a range of factors. These may include: factors relating tothe credibility of evaluation such as data quality, perceptions among practitioners ofwhether evaluation methods are valid ways of generating knowledge and the extentto which evaluators are trusted; the ways in which evaluation findings are presentedand disseminated that are accessible, timely and meet the needs of the audience;the relevance and usefulness of evaluation findings to policy makers andpractitioners; and the abilities of policy makers and practitioners to act on evaluationfindings. These may be influenced by alternative and sometimes conflictingpressures that influence their decision making, such as changing policy directions atnational level, policy makers and practitioners’ abilities to interpret and apply findings,systems and institutional cultures of accepting and acting on evidence and policymakers and practitioners’ abilities to act on evaluation findings within budgetaryconstraints (McDonald, 2000; Davies, et al., 2000; Percy-Smith, et al., 2000; Becker& Bryman, 2004; Coote, et al., 2004).Table 1: Roles and orientations of social policy evaluationGeneratinglearning/informingorganisations’ developmentPerformance managementConsultant/technician/analystEmpowerment/capacitybuildingevaluationResearch-orientatedevaluationTheory-driven evaluationInclusive evaluationGenerating formative learning to helporganisations develop throughout their lives.Understanding what works and why; acting as acritical friendDemonstrating and showing accountability forimpact/success/failure of an organisation’sactivitiesProviding solutions to organisations’problems/questions; analysis of secondary data onbehalf of an organisationFacilitating organisations’ reflection and supportingthe capacity for self-evaluation/monitoringDeveloping and expanding practitioners/policymakers’ understandings of conceptsThe explicit use of a theoretical position thatinforms analysis/interpretation of evaluationmaterialAttempts to redress power imbalances andinequalities in society; sees communities asprinciple interest groups of evaluation4Introduction
Page 1 and 2: RESEARCHLocal Evaluation ofChildren
Page 4 and 5: When citing this report please show
Page 6 and 7: ContentsExecutive SummaryIIntroduct
Page 8 and 9: Local Evaluation of the Children’
Page 10 and 11: 10. Local evaluators highlighted th
Page 12 and 13: 20. Some evaluators involved childr
Page 14 and 15: successes of partnerships’ work (
Page 16: 38. Conditions reinforcing effectiv
Page 19: Children’s Fund initiative, namel
Page 23 and 24: starting points and timetables of d
Page 25 and 26: 1. Process evaluation: critical ass
Page 27 and 28: time commitments and pressures amon
Page 29 and 30: tantamount to auditing when origina
Page 31 and 32: to a certain extent, because I was
Page 33 and 34: acknowledge that findings presented
Page 35 and 36: methods that were seen as being app
Page 38 and 39: Chapter 2: Methods Adopted by Child
Page 40 and 41: place. In one Wave One evaluation t
Page 42 and 43: provided by projects weakened the a
Page 44 and 45: confidently attribute any trends in
Page 46 and 47: A Wave Two evaluation report descri
Page 48 and 49: • Recruitment of the researcher:
Page 50 and 51: drawn on by evaluators include semi
Page 52 and 53: Chapter 3: How Local Evaluation Sup
Page 54 and 55: participation and work towards the
Page 56 and 57: highlighted in Table 1). These incl
Page 58 and 59: of had an impact. So we used a case
Page 60 and 61: Similarly an evaluator of a Wave Tw
Page 62 and 63: Evaluators sometimes struggled to a
Page 64: 6. A number of factors inhibit eval
Page 67 and 68: A number of evaluations described h
Page 69 and 70: 4.2 Key Messages from Local Evaluat
Page 71 and 72:
for staff, typically dedicated part
Page 73 and 74:
Where partnership working was descr
Page 75 and 76:
strategic changes in children’s s
Page 77 and 78:
9. Evaluators reported that some co
Page 79 and 80:
The importance of providing finding
Page 81 and 82:
National Evaluation of the Children
show all

Local Evaluation of Children's Services Learning from the Children's ...

Create successful ePaper yourself

Delete template?

Save as template?