Evaluation: Principles, Methodology, Instruments

instruments made available centrally). In addition, there is internal evaluation byexternal individuals, if e.g. graduates assess their training programme or school.Finally, selected quality objectives are specified also externally for cross-schoolevaluation.The centrally developed evaluation instruments partly cover various school types (e.g.in the survey subject “performance assessment”) and are partly tailored to the requirementsof the respective school type (e.g. topicality of teaching contents). Evaluation instrumentsused include, in particular, questionnaires (for different target groups), interview guidelines(mainly for people external to school), and survey sheets (mainly for school itself). Surveyunits are both people (e.g. students, graduates, teachers, parents, experts) andorganisational units (e.g. schools, companies). Particular attention is paid to ensure theanonymity of those surveyed; no data are collected that would reveal, either by themselvesor in combination with other data, the identity of a person.The implementation and evaluation of the interviews is supported by making availablea comprehensive technical infrastructure. All survey instruments may be edited as web formsvia the Internet 1 . The data are stored on a central server; upon completion of the interviewsan automatic standard evaluation is conducted, which shows for every question, as well asthe number of measuring values on which it is based, the result in the form suited for therespective answer format (e.g. mean value, percentage value of an answer alternative,frequency distribution of answer options or rating levels, free text entries). A specimen of astandard evaluation is shown in Fig. 1 for a rating question that has been taken from thestudent questionnaire on the quality objective “to make performance assessmenttransparent” (N designating the number of students, M and SD the arithmetic means and thestandard deviation of given ratings, respectively, and M s the mean of the means taken fromschools/colleges).The central database allows reference values to be determined in a simple way so thatan educational institution can see its own results in a wider context. The database isorganised such that access to person-related data is in principle not possible and that everyeducational establishment can see only its own data, in an aggregated form in the standardevaluation. For selected questions and, in the event data are released by the schoolaccordingly, data are accessible also directly to the regional school inspection in the provincevia the database. The Federal Ministry receives overall results of the provinces for selectedquestions. Fig. 2 gives an overview of dataflow and access rights to the results by theexample of an evaluation at school level.1 Finished instruments are accessible to schools on the QIBB website www.qibb.at.2/3