User Guide for the TIMSS International Database.pdf - TIMSS and ...

A C H I E V E M E N T S C O R E S C H A P T E R 6
The national Rasch scores should not be used for international comparisons for two
reasons: they were computed with a different set of item difficulties for each country, and
the weighted mean score within each country is always equal to 150. National Rasch
scores can be found in the Student Background data files and in the Written Assessment
data files.
ASMPV01-ASMPV05 Mathematics Plausible Value 1 to Plausible Value 5 –
Population 1
ASSPV01-ASSPV05 Science Plausible Value 1 to Plausible Value 5 – Population 1
BSMPV01-BSMPV05 Mathematics Plausible Value 1 to Plausible Value 5 –
Population 2
BSSPV01-BSSPV05 Science Plausible Value 1 to Plausible Value 5 – Population 2
As described in chapter 5, TIMSS made use of multiple imputation or “plausible values”
methodology to provide estimates of student proficiency in mathematics and science.
Because of the error involved in the imputation process, TIMSS produced not one but
five imputed values for each student in mathematics and science. The need for plausible
values arises from the fact that any student was administered only a fraction of the items
in the assessment, as described in Chapter 2. Time constraints did not allow for all the
items to be administered to each student. A plausible value is an estimate of how the
individual student would have performed on a test that included all possible items in the
assessment (see Chapter 5). Since no student responded to all items, this estimate is based
on the responses to the items that were included in the test booklet that the student
actually took and the performance of students with similar characteristics.
Plausible values have been shown to improve the estimation of population parameters.
They were developed during the analysis of the 1983-84 NAEP data in order to improve
estimates of population distributions. The general theory of the NAEP plausible values
can be attributed to Mislevy (Mislevy and Sheehan, 1987; 1989) based on Rubin’s work
(Rubin, 1987) on multiple imputations.
Within a subject area and across the sample, one set of plausible values can be considered
as good as another. Each of these sets is equally well designed to estimate population
parameters, although the estimates will differ somewhat. This difference is attributable to
imputation error. Five sets of plausible values are provided so that analyses may be
replicated as many as five times. Results which vary from replication to replication may
be influenced by unreliability in the achievement measures, and considered to be suspect.
In the TIMSS international reports, the reliability of the achievement measures, as
reflected in the inter correlations between the five plausible values was found to be
sufficiently high that the imputation error could be ignored. For the purpose of reporting
international achievement, therefore, only the first plausible value was used. However, all
five values are provided in the International Database for use by other analysts. The
plausible values are included in the Student Background data files and in the Student-
Teacher Linkage files.
T I M S S D A T A B A S E U S E R G U I D E 6 - 3