User Guide for the TIMSS International Database.pdf - TIMSS and ...

C H A P T E R 4 D A T A C O L L E C T I O N P R O C E D U R E S
The Survey Operations Manual for Populations 1 and 2 was prepared by the IEA Data
Processing Center for the NRCs and their colleagues who were responsible for implementing
the TIMSS data collection and processing procedures. It describes the activities and
responsibilities of the NRCs from the moment the international testing materials arrived at the
national center to the moment the cleaned data sets and accompanying documentation were
sent to the IEA Data Processing Center. In addition to detailed within-school sampling
instructions, the manual included:
· Procedures for translating and assembling the test instruments and questionnaires
· Instructions for obtaining cooperation from the selected schools
· Explicit procedures for packing and sending materials to the schools
· Preparations for test administration
· Instructions for data entry and verification
Included in this manual were a set of Survey Tracking Forms that were completed at various
stages of the study to track schools, classrooms, and students, and ensure proper linkage
among them in the database.
Each school was asked to appoint a coordinator to be the liaison between the national
research center and the school. The School Coordinator Manual describes the steps the
School Coordinator followed once the testing materials arrived at the school until they were
returned to the NRC. Essentially, the School Coordinator was responsible for receiving the
shipment of testing materials from the national center and providing for their security before
and after the test date. The School Coordinator administered the teacher questionnaires,
arranged the testing accommodations, trained Test Administrators, and arranged for make-up
sessions when necessary.
The Test Administrator Manual covered the procedures from the beginning of testing to the
return of the completed tests, questionnaires, and tracking forms to the School Coordinator.
The manual includes a Test Administration Script to be read by the Test Administrator. The
Test Administrators were responsible for activities preliminary to the testing session, including
maintaining the security of the test booklets and ensuring adequacy of supplies and the
testing environment. Activities during the testing session, included distribution of the test
booklets to the appropriate students (using the Student Tracking Form), timing of the testing
and breaks, and accurately reading the test administration script.
4.2 Free-Response Scoring
Upon completion of each testing session, the School Coordinator shipped the booklets,
questionnaires, and forms to the National Research Center. The NRCs then organized the
instruments for scoring and data entry. These procedures were designed to maintain
identification information that linked students to schools and teachers, minimize the time and
effort spent handling the booklets, ensure reliability in the free-response scoring, and
document the reliability of the scoring. Since approximately one-third of the written test time
and all of the performance assessment time was devoted to free-response items, the freeresponse
scoring was complex.
4 - 2 T I M S S D A T A B A S E U S E R G U I D E