Local Evaluation of Children's Services Learning from the Children's ...

More documents

Recommendations

Info

confidently attribute any trends in this data to the work of the X (project) as otherfactors may be affecting the figures’.The evaluation then analysed child level data based on twelve randomly selectedchildren in the schools targeted by the project focusing on school attendance. At thislevel the evaluation report appears able to identify and evidence significant impactsof the project work with more confidence. For example, one pupil’s attendance levelsincreased by 40% over a seven month period, during which s/he had receivedsupport from the project: ‘…it is clear that without the support and encouragement ofthe X (Project), all of these children and probably many others, would have missed alarger percentage of their early years education’. There were also attempts tomeasure the impacts of the reading project at individual child level by considering anational pupil survey conducted by the organisation operating the project. The surveymeasured impacts in terms of ‘attitudes to reading’ and demonstrated animprovement in this during 2002. Evaluators detailed the improved reading ages offive randomly chosen pupils at one of the targeted schools, as described by the staffdelivering the project. Qualitative impacts were also described. These were based onsome one-to-one interviews with children accessing the service and duringevaluation events where children were encouraged to express their thoughts andopinions on the projects they had been accessing through art work and writing.Analysis of such data led evaluators to note the development of ‘supportiverelationships built between the children and the project volunteers’ and ‘the effort… toincorporate the children’s views on how their work should progress and particularlywhat books they enjoyed’.It therefore appears that it was when the evaluator reported upon data gathered fromchild-level analysis that impacts could be described with greater confidence. Thesequalitative data typically referred to the experiences of individual children rather thanbeing of a quantitative nature. There remained, however, potential difficulties withthese data. For example, the selection of the five pupils by those providing theservice may not have resulted in a typical sample.Overall, however, the evaluators argued that their multiple methods approach(analysis of quantitative data at national, local, school and individual child-levels,together with qualitative interviews allowing for service providers’ and service users’perceptions of project impacts to be revealed) whilst imperfect, allowed them toChapter 2 27
suggest with some confidence that Children’s Fund projects had led to positiveoutcomes for children using the services.2.3 Evaluating ProcessesMany local evaluation reports undertook assessments of key Children’s Fundprocesses associated with the delivery of programmes including: setting targets andcommissioning services; processes supporting the infrastructure of the programme,including communications arrangements, financial management arrangements andcross project information and referral arrangements; processes in place to report onprogress, including monitoring and evaluation arrangements; work on establishingnew models of working among mainstream agencies; children and young people’sparticipation and the effectiveness of strategic partnership arrangements.A range of methods tended to be used by individual evaluators. A significant numberof reports describe the use of data elicited from in-depth qualitative interviews orfocus groups with strategic stakeholders when discussing perceptions andexperiences relating to the effectiveness of strategic processes developed by localChildren’s Funds. The focus for interviews was typically the experiences and learningindividual members of partnership boards could offer, with some evaluators carryingout interviews with stakeholders in the wider strategic landscape in the locality, forexample key individuals in Social Services or large voluntary organisations who maybe affected by Children’s Fund activity.Other methods adopted include observational work within strategic and in projectsettings, quantitative questionnaire surveys to establish the systems and strategies inplace between stakeholders and documentary analysis of key partnership planningdocuments and board minutes. A Wave Three evaluator, for example, described howdrawing on observational, interviews and focus group work at both strategic andoperational levels of Children’s Fund activity enabled a more detailed, more holisticpicture of partnership working, prevention and participation to emerge:I think it gave me a holistic picture of everything for a start because of courselike I say, the ‘three Ps’ [prevention, participation and partnership], I feel arealmost concepts as well as… they’re almost virtual… and people have differentunderstandings in different areas, and may have different understandings of it.So it’s about looking at their understanding, or their interpretation of preventionand participation, and partnership working. They gave me a more holisticunderstanding of that, and shaped that.28Chapter 2
Page 1 and 2: RESEARCHLocal Evaluation ofChildren
Page 4 and 5: When citing this report please show
Page 6 and 7: ContentsExecutive SummaryIIntroduct
Page 8 and 9: Local Evaluation of the Children’
Page 10 and 11: 10. Local evaluators highlighted th
Page 12 and 13: 20. Some evaluators involved childr
Page 14 and 15: successes of partnerships’ work (
Page 16: 38. Conditions reinforcing effectiv
Page 19 and 20: Children’s Fund initiative, namel
Page 21 and 22: engaging with policy makers, practi
Page 23 and 24: starting points and timetables of d
Page 25 and 26: 1. Process evaluation: critical ass
Page 27 and 28: time commitments and pressures amon
Page 29 and 30: tantamount to auditing when origina
Page 31 and 32: to a certain extent, because I was
Page 33 and 34: acknowledge that findings presented
Page 35 and 36: methods that were seen as being app
Page 38 and 39: Chapter 2: Methods Adopted by Child
Page 40 and 41: place. In one Wave One evaluation t
Page 42 and 43: provided by projects weakened the a
Page 46 and 47: A Wave Two evaluation report descri
Page 48 and 49: • Recruitment of the researcher:
Page 50 and 51: drawn on by evaluators include semi
Page 52 and 53: Chapter 3: How Local Evaluation Sup
Page 54 and 55: participation and work towards the
Page 56 and 57: highlighted in Table 1). These incl
Page 58 and 59: of had an impact. So we used a case
Page 60 and 61: Similarly an evaluator of a Wave Tw
Page 62 and 63: Evaluators sometimes struggled to a
Page 64: 6. A number of factors inhibit eval
Page 67 and 68: A number of evaluations described h
Page 69 and 70: 4.2 Key Messages from Local Evaluat
Page 71 and 72: for staff, typically dedicated part
Page 73 and 74: Where partnership working was descr
Page 75 and 76: strategic changes in children’s s
Page 77 and 78: 9. Evaluators reported that some co
Page 79 and 80: The importance of providing finding
Page 81 and 82: National Evaluation of the Children

Local Evaluation of Children's Services Learning from the Children's ...

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?