User Guide for the TIMSS International Database.pdf - TIMSS and ...

D A T A C O L L E C T I O N P R O C E D U R E S C H A P T E R 4
The background questionnaires were stored with the various tracking forms so that the data
entry staff could control the number of records to enter and transcribe the necessary
information during data entry. NRCs were asked to arrange for double-entry of a random
sample of at least 5% of the test instruments and questionnaires. An error rate of 1% was
considered acceptable.
After entering data files in accordance with the international procedures, countries submitted
their data files to the IEA Data Processing Center.
4.4 Database Creation
Even though extreme care was taken in developing manuals and software for use by the more
than 40 participating countries, the national centers inadvertently introduced various types of
inconsistencies in the data, which needed to be thoroughly investigated by the IEA Data
Processing Center and the International Study Center at Boston College.
The TIMSS data underwent an exhaustive cleaning process designed to identify, document,
and correct deviations from the international instruments, file structures, and coding schemes.
The process also emphasized consistency of information with national data sets and
appropriate linking among the many data files. The national centers were contacted regularly
throughout the cleaning process and were given multiple opportunities to review the data for
their countries.
4.5 Instrument Deviations and National Adaptations
Ensuring the international comparability of both the cognitive and contextual variables was
an important aspect of TIMSS. A number of data management steps were focused on
evaluating the international comparability of the TIMSS items, and any deviations for specific
items were handled on an individual basis. An overview of the procedures and policies
applied to ensuring international comparability is provided in the following sections relating
to the test items and the background questionnaire items.
4.5.1 Cognitive Items
All TIMSS written assessment test items and performance assessment tasks were originally
developed in English and then translated by the participating TIMSS countries into more than
30 languages. In addition to the translation verification steps used for all TIMSS test items
(Maxwell, 1996), a thorough item review process was also used to further evaluate any items
that were functioning differently in different countries according to the international item
statistics (Mullis and Martin, 1997). As a result of this review process, a few items were
identified as not being internationally comparable in certain countries and were deleted from
the international data files and from the analyses for the international reports. A list of all
these deleted items as well as any items omitted in the test instruments for specific countries is
given in Table 4.2.
T I M S S D A T A B A S E U S E R G U I D E 4 - 7