here - TIMSS and PIRLS Home - Boston College

312 appendix a: overview of pirls 2006 procedures
this, there is a danger that results may be distorted by a “floor” effect—that the
achievement of low-performing students may be overestimated because there
are not sufficient items matched to their ability levels. Even though the PIRLS
scaling approach is very robust, there is concern that achievement results
based on very low average performance may not be sufficiently accurate
to report. Based on examinations of the data, PIRLS 2006 used a guideline
of not reporting scaled achievement results for countries with an average
percent correct of 20 percent or less on the set of items comprising the
scale. Accordingly, results on the interpreting, integrating, and evaluating
scale were not reported for Kuwait, Morocco, Qatar, and South Africa.
Estimating Sampling Error
Because the statistics presented in this report are estimates of national
performance based on samples of students—rather than on the values that
could be calculated if every student in every country had answered every
question—it is important to have measures for the degree of uncertainty of
the estimates. The jackknife procedure was used to estimate the standard
error associated with each statistic presented in this report. As well as
sampling error, the jackknife standard errors also include an error component
due to variation between the five plausible values generated for each student.
The use of confidence intervals (based on the standard errors) provides
a way to make inferences about the population means and proportions in a
manner that reflects the uncertainty associated with the sample estimates.
An estimated sample statistic plus or minus 2 standard errors represents a
95 percent confidence interval for the corresponding population result.
Reporting Student Achievement at the PIRLS 2006
International Benchmarks
To provide richly descriptive information about what performance on the
PIRLS reading scale means in terms of the reading skills that students have
and the comprehension processes and strategies they can bring to bear, PIRLS
identified four points on the scale for use as international benchmarks of
For further discussion, see Foy, P., Galia, J., & Li, Isaac. (2007). Scaling the PIRLS 2006 reading assessment data. In M.O. Martin,
I.V.S. Mullis, & A.M. Kennedy (Eds.), PIRLS 2006 technical report. Chestnut Hill, MA: Boston College.
Procedures for computing jackknifed standard errors are presented in Kennedy, A.M. & Trong, K. L. (2007). Reporting PIRLS 2006
student achievement in reading. In M.O. Martin, I.V.S. Mullis, & A.M. Kennedy (Eds.), PIRLS 2006 technical report. Chestnut Hill, MA:
Boston College.