Evaluation: Principles, Methodology, Instruments

More documents

Recommendations

Info

instruments made available centrally). In addition, there is internal evaluation byexternal individuals, if e.g. graduates assess their training programme or school.Finally, selected quality objectives are specified also externally for cross-schoolevaluation.The centrally developed evaluation instruments partly cover various school types (e.g.in the survey subject “performance assessment”) and are partly tailored to the requirementsof the respective school type (e.g. topicality of teaching contents). <strong>Evaluation</strong> instrumentsused include, in particular, questionnaires (for different target groups), interview guidelines(mainly for people external to school), and survey sheets (mainly for school itself). Surveyunits are both people (e.g. students, graduates, teachers, parents, experts) andorganisational units (e.g. schools, companies). Particular attention is paid to ensure theanonymity of those surveyed; no data are collected that would reveal, either by themselvesor in combination with other data, the identity of a person.The implementation and evaluation of the interviews is supported by making availablea comprehensive technical infrastructure. All survey instruments may be edited as web formsvia the Internet 1 . The data are stored on a central server; upon completion of the interviewsan automatic standard evaluation is conducted, which shows for every question, as well asthe number of measuring values on which it is based, the result in the form suited for therespective answer format (e.g. mean value, percentage value of an answer alternative,frequency distribution of answer options or rating levels, free text entries). A specimen of astandard evaluation is shown in Fig. 1 for a rating question that has been taken from thestudent questionnaire on the quality objective “to make performance assessmenttransparent” (N designating the number of students, M and SD the arithmetic means and thestandard deviation of given ratings, respectively, and M s the mean of the means taken fromschools/colleges).The central database allows reference values to be determined in a simple way so thatan educational institution can see its own results in a wider context. The database isorganised such that access to person-related data is in principle not possible and that everyeducational establishment can see only its own data, in an aggregated form in the standardevaluation. For selected questions and, in the event data are released by the schoolaccordingly, data are accessible also directly to the regional school inspection in the provincevia the database. The Federal Ministry receives overall results of the provinces for selectedquestions. Fig. 2 gives an overview of dataflow and access rights to the results by theexample of an evaluation at school level.1 Finished instruments are accessible to schools on the QIBB website www.qibb.at.2/3
Access to infrastructure is regulated via carefully designed user administration. Forsurveys covering students, graduates, teachers, parents and administrative staff, activationcodes are given. Every person involved receives a TAN that may be used only once.Figure 1: Example of astandard evaluationSchoolRelease ofdefined standardevaluationsRegionallevelCentrallevelRelease of an evaluationinstrument for a targetgroup (TAN)Retrieval ofstandardevaluationsWeb applicationData pool ofschoolCentraldatabaseFigure 2: Organisation of theweb-based processing ofevaluation dataInternetStudents, graduates, teachers, patrents, ...3/3
Page 1: Evaluation:Principles, Methodology,

Evaluation: Principles, Methodology, Instruments

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?