ICCS 2009 Technical Report - IEA

SummaryThe jackknife repeated replication technique (JRR) was applied in order to allow reportingof sampling errors in ICCS reports. Plausible value methodology was used with respect toreporting civic knowledge scores. This process permitted estimation not only of variance due tosampling but also of imputation variance.Different types of significance test were used to compare means or percentages betweenparticipating countries, with the ICCS average, or between subgroups within the sample. Theequating (or link) error was taken into account when averages of civic content knowledge in2009 were compared with averages of civic content knowledge from the CIVED survey of1999.ICCS 2009 data were used to estimate the multiple regression models as well as the hierarchicallinear models, and explained variance decomposition was used to estimate the uniquecontribution of different sets of predictor variables in the multiple regression models. Theexplained variance at student and school levels was compared separately whenever two-levelhierarchical linear models were used.Missing data problems became more prevalent during multivariate analyses of ICCS data thatinvolved larger numbers of predictor variables. For the reported analyses, data were treatedby adding missing indicators and substituting missing values with modes or means. Anyoneconducting multivariate analysis of ICCS data needs to take missing data problems intoaccount and should also explore possibilities for applying more advanced methods, includingimputation procedures.ReferencesCohen, J., & Cohen, P. (1975). Applied multiple regression/correlation analysis for the behavioralsciences. Hillsdale, NJ: Lawrence Erlbaum Associates.Gonzalez, E. J., & Foy, P. (2000). Estimation of sampling variance. In M. O. Martin, K. D.Gregory, & S. E. Semler (Eds.), TIMSS 1999: Technical report. Chestnut Hill, MA: Boston College.Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysismethods. Newbury Park, CA: Sage Publishers.Raudenbush, S. W., Bryk, A. S., Cheong, Y. F., & Congdon, R. (2004). HLM 6: Hierarchicallinear and nonlinear modeling. Chicago, IL: Scientific Software International.Schulz, W., Ainley, J., Fraillon, J., Kerr, D., & Losito, B. (2010). ICCS 2009 international report:Civic knowledge, attitudes, and engagement among lower-secondary school students in 38 countries.Amsterdam, the Netherlands: International Association for the Evaluation of EducationalAchievement (IEA).Westat (2007). WesVar ®4.3: User’s guide [computer software]. Rockville MD: Author.Wolter, K. M. (1985). Introduction to variance estimation. New York: Springer.280ICCS 2009 technical report