Local Evaluation of Children's Services Learning from the Children's ...

It is not possible to accurately quantify levels and types engagement of children andyoung people, parents and carers in local evaluations, although analysis of reportssuggests that around half do not involve these groups. Of the rest, most involve themas interview/questionnaire/focus group respondents. A minority of evaluations appearto be involving them in the design stages, undertaking data collection or analysingevaluation data. Nevertheless, there are some examples of local evaluations thathave extensively involved children and young people. One such evaluation of aWave Three partnership that conducted a piece of work on uptake of services,involved children and young people in recruiting the researcher, designing theevaluation, gathering data and analysing findings. See Chapter 2 of this report for areview of the means adopted by some evaluators to engage children and youngpeople in various aspects of evaluations.The credibility of methods and evidenceA number of evaluators refer to the importance of adopting a defensible evaluationdesign, that is the evaluation is conducted rigorously and systematically in terms ofmethods of data collection, analysis and interpretation and that methods adopted areappropriate in answering particular questions evaluations set out to address. Somereports stress the importance of making credible claims and ensuring these are fullysupported by the data. Other reports state that the local evaluations were based onthe openness and transparency of the evaluation process and the methods used interms of data collection, analysis and interpretation of data. Hence, messagescontained in evaluation outputs should be fully supported by evidence. A broadly heldview is that evaluation methods should reflect the particular aspects of a partnershipthat evaluations are examining; this is reflected in use of multiple methods localevaluations use across different aspects of their work. Other reports emphasise theimportance of a broad-based view of partnerships both in terms of examining asmany aspects of partnerships’ work as possible and situating this work within localand national policy contexts and academic literature. The ways evaluation methodshave been used in combination in examining aspects of partnerships’ work andaddressing particular questions are explored in Chapter 2 of this report.Ethical approaches to evaluationA number of evaluation reports allude to ethical considerations that have guided theconduct of gathering data. Such considerations include the importance of beingculturally and linguistically sensitive in working with diverse groups and to berespectful to evaluation participants. Some evaluators indicated that they had appliedChapter 1 17