Peer Review Impact Analysis Report - Peer Review in European VET

Peer Review Impact Analysis Report 6The conduct of Peer Review following the Manual is supported by forms for reporting, checklists andquality areas compiled in a “Tool-box” and both web-based and face-to-face Peer Training. The Toolboxalso contains a peer application form which solicits extensive information on potential Peers andask for their self-assessment in key competence areas. Report forms introduce some uniformity intothe procedure allowing for cross-comparisons between Peer Reviews.The pilot phases were subject to constant monitoring by the project management: All steps in theprocedure were documented, the observation of critical quality standards by VET Providers andPeers was monitored (Peer applications, reports of VET providers) and reflective statements andwritten feedback of all participants collected and evaluated. In the first pilot phase an externalevaluation was carried out in addition to the internal monitoring and evaluation. The evaluation findingswere used to improve the European Peer Review procedure.For the research on the impact of the Peer Review, all in all fourteen Peer Reviews were included (cf.below sample).2 Methodological approach, theoretical model andquality assurance2. 1 Evaluation design and methodsThe research follows a qualitative case study design.Case StudiesAs has been stated above, one of the main assets of the European Peer Review is that it gives practicalguidelines and quality standards but – within these quality requirements – allows for tailoring theprocedure to the specific situation, the aims and needs, and the organizational culture of the particularVET Provider. Thus considerable variation in implementation is possible. In a European context,national quality systems, institutional quality management, evaluation and management culturesvaried and so did, to a certain extent, the Peer Reviews. An investigation of the uses and impacts ofPeer Review thus has to refer to the actual implementation. The case study design was chosen becauseit allows for a consideration of the variations in the conduct of the Peer Review.Qualitative interviewsQualitative research relies on verbal data which may also (but need not) be translated into quantitativedata, where appropriate. Instead of looking for selected and usually narrow data on certain phenomenawhich then can be analysed statistically to prove or falsify hypotheses, qualitative researchlooks for a rich description of human experience. It is especially appropriate for use in fields or subjectswhere comprehensive theories which could underlie a quantitative approach are missing or notvery well developed. It is also the method of choice for discovering new theories or enlarging existingones, i.e. to help understand phenomena. It is therefore highly appropriate for the exploratory characterof this study. It is also the approach used in Peer Review in order to fully grasp what is happeningin the VET institution and to help make sense of quantitative data and indicators (which usuallyindicate where we are but not why we are there).The case studies therefore relied on qualitative interviews with different stakeholders in the VETProviders concerned: Management of the unit reviewed (mandatory), also middle management ifexistent; Peer Review Facilitators, quality managers, teachers, students involved in the Peer Reviews,if possible, teachers/staff/(students) not involved in the Peer Review, other staff / stakeholders whowere involved as appropriate.Gutknecht-Gmeiner 2010Peer Review Impact2009-1-FI1-LEO05-01584