Peer Review Impact Analysis Report - Peer Review in European VET

Peer Review Impact Analysis Report 41 Introduction1. 1 BackgroundIn the European Peer Review projects – Peer Review in initial VET AT/04/C/F/TH-82000, Peer ReviewExtended EAC/32/06/13 (LE-78CQAF) and Peer Review Extended II LLP-LdV/TOI/2007/AT/0011 –, thePeer Review methodology prevalent in higher education has been transferred and tailored to VET.The projects have been very successful with 15 European countries taking part between 2004 and2009 and 25 transnational Peer Reviews carried out in three pilot phases. Peer Review is currentlybeing introduced as a new tool for quality assurance in VET in countries such as Austria, Finland, Italy,Hungary, Spain, Norway etc. Furthermore, there have been efforts to establish a sustainable networkand structure for Peer Reviews on a transnational European level.One of the most prominent promises of Peer Review − also in comparison to other external evaluations− is its impact in terms of stimulating improvement. Studies have shown that in quality managementthe fourth phase of the quality cycle (Plan- Do- Check- Act) is in fact the weakest, i.e. thatvaluable evaluative information is not used, or only to a low degree.Evaluation of the pilot phases during the Peer Review projects primarily focused on the implementationof the procedure, its practicability and acceptability. In the light of further implementation ofPeer Review on national (and perhaps also international level) it thus still needs to be clarified 1)whether Peer Review in fact supports further action and leads to improvements in VET institutionsand 2) how these effects, if at all, come about.1. 2 Aims and scope of the researchDuring the Peer Review projects, data on usability and usefulness of Peer Review for VET institutionshave been collected through monitoring, evaluations and in partner meetings and presentations atinternational conferences (esp. in the International Peer Review Conferences in Pécs 2007 and Lisbon2009 respectively). Some Peer Review partner institutions have also publicly presented the measuresthey have taken after the Peer Review. Yet, comprehensive data on the use of Peer Review had so farnot been available.In the project Peer Review Impact, a thorough investigation and analysis of a sample of the 25 PeerReviews carried out between 2006 and 2009 was carried out to• Check whether Peer Review actually has had an impact on the reviewed institutions and whatkind of impact it is• Understand how this impact comes about• And to distil critical success factors for Peer Review implementation from these findings.This called for a meta-evaluation of the pilots to determine “programme fidelity”, i.e. to what extentquality requirements of the Peer Review procedure relevant for further use of Peer Review resultshave been observed. Furthermore, the actual use of Peer Review (use of findings, but also processus) needed to be explored and instrumental use evaluated. This then should lead to the identificationof critical success factors to optimise evaluation use and impact.Gutknecht-Gmeiner 2010Peer Review Impact2009-1-FI1-LEO05-01584