Final Report of the Evaluation of the Pupil Learning Credits Pilot ...

More documents

Recommendations

Info

9 analysed as part of the evaluation of the PLC pilot scheme. 10 The samples of pupils from comparison schools were drawn from schools in EiC Phase 2 and Phase 3 areas 11 that, in January 2000, had 35 per cent or more pupils on roll who were known to be eligible for Free School Meals. The pupil survey formed part of the evaluation of the Excellence in Cities policy, 12 and thus covered a wide range of areas and topics. For the purpose of this report, pupils’ answers to questions of particular relevance to the PLC pilot scheme have been analysed. That is, items relating to participation in additional educational activities that may enrich and add depth to pupils’ learning are presented. As noted earlier, the PLC pilot scheme was intended to promote inclusion and to provide pupils with the means to improve their levels of attainment through, for example, raising self-confidence and self-esteem and improving attitudes to learning. Responses of pupils in PLC schools to items relating to these objectives are therefore compared with those of pupils in comparison schools. This form of comparison was not only intended to assess whether pupils in schools receiving PLC funds reported participating in more enrichment activities but also to provide insight into possible mechanisms through which any improvement in attainment (identified through the multi-level modelling) might be being delivered. It is, however, important to note that although the analyses identify statistically significant associations they do not establish causation; thus differences between PLC and comparison schools should not be assumed to arise from the PLC pilot scheme. 2.4 Statistical analyses of national pupil-level data Two sets of quantitative analyses were carried out for the evaluation. Multi-level modelling Pupils’ academic progress is examined by means of multilevel modelling using the national pupil-level (also called value-added) datasets for 2001, 2002 and 2003. Multilevel modelling is a form of regression analysis that takes account of the fact that data are clustered. By allowing for a hierarchical structure of the data, a multilevel modelling approach enables us to control for background factors at different levels (in our case, at individual pupil, school and LEA level). Annex A provides the list of background variables included in the model. 10 11 12 The PLC pilot scheme was particularly targeted at pupils in Key Stage 3, but the telephone interviews with headteachers and the case studies of schools revealed that many schools decided to also target the funding on other year groups (see Braun and West, 2004); for this reason, it was decided to focus in the analysis on pupils in Years 9 and 11. EiC Phase 2 LEAs: Barking and Dagenham, Brent, Bristol, Ealing, Gateshead, Halton, Hartlepool, Kingston upon Hull, Leicester City, Middlesbrough, Newcastle, North Tyneside, Nottingham, Redcar and Cleveland, Rochdale, St Helens, part of Sefton, Solihull, South Tyneside, Stockton-on-Tees, Stoke-on-Trent, Sunderland and Wirral. EiC Phase 3 LEAs: remaining part of Sefton, Sandwell, Hounslow, Wolverhampton, Oldham, Barnsley, Doncaster, Luton, Blackburn with Darwen, Enfield and Blackpool. The evaluation was carried out by a consortium comprising the NFER, the LSE and the Institute for Fiscal Studies (see Kendall et al., 2005; NFER, 2005).
10 As part of the evaluation of Excellence in Cities, national value-added data linking Key Stage 3 (KS3) and Key Stage 4 (KS4) results in 2003 to prior attainment and information from the Pupil Level Annual School Census (PLASC) were modelled 13 in order to assess the possible impact of EiC on pupils’ progress during Key Stage 3 and Key Stage 4. Given that the PLC pilot scheme targeted additional resources on particular schools that were also part of the Excellence in Cities Policy, statistical analyses were undertaken in order to establish any associations between the PLC pilot scheme and various attainment outcomes, having controlled for other variables. Economic evaluation The economic evaluation involved analyses of the effect of the PLC pilot scheme on pupil attainment and school attendance. It uses a ‘difference-in-differences’ approach that compares PLC and comparison group schools before and after the pilot scheme (i.e., outcomes for 2003 relative to 2001), whilst controlling for changes in a set of background variables known to be associated with attainment. Two comparison groups are used, non-PLC schools in areas designated as EiC Phase 1 and non-PLC schools in areas designated as EiC Phase 1 or Phase 2. A cost-benefit analysis is also presented which considers the possible impact of any increase in attainment resulting from the PLC pilot scheme on subsequent labour market earnings. 13 This modelling was carried out by Ian Schagen at NFER and we are grateful to him for allowing us to present extracts from his analysis in this report.
Page 1 and 2: RESEARCH Final Report of the Evalua
Page 3 and 4: 2 TABLE OF CONTENTS EXECUTIVE SUMMA
Page 5 and 6: 4 where pupils in PLC schools repor
Page 7 and 8: 6 1. INTRODUCTION The Pupil Learnin
Page 9: 8 2. METHODS This report draws toge
Page 13 and 14: 12 curricular activities, departmen
Page 15 and 16: 14 in ten respondents stated that t
Page 17 and 18: 16 Main strengths and weaknesses of
Page 19 and 20: 18 not only intended to assess whet
Page 21 and 22: 20 • pupils in PLC schools report
Page 23 and 24: 22 School experience and behaviour
Page 25 and 26: 24 Around four out of ten pupils re
Page 27 and 28: 26 We examined whether there were s
Page 29 and 30: 28 Aspirations for the future Pupil
Page 31 and 32: 30 Multi-level modelling Summary Th
Page 33 and 34: 32 Pupils’ progress from KS2 to G
Page 35 and 36: 34 Difference-in-differences and ec
Page 37 and 38: 36 when using the discrete measures
Page 39 and 40: 38 4. SUMMARY AND POLICY IMPLICATIO
Page 41 and 42: 40 In particular, the PLC pilot sch
Page 43 and 44: 42 MCNALLY, S. (2005) Economic Eval
Page 45 and 46: ANNEX B CHARACTERISTICS OF PLC SURV

Final Report of the Evaluation of the Pupil Learning Credits Pilot ...

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?