ICCS 2009 Technical Report - IEA

The software package HLM 6.08 (Raudenbush, Bryk, Cheong, & Congdon, 2004) was usedto estimate all hierarchical models. Countries that did not meet IEA sample participationrequirements were excluded from the analyses, as were countries where there were fewer than50 schools. The countries to which these provisos applied were Hong Kong SAR, Liechtenstein,Luxembourg, and the Netherlands.In most countries, one intact classroom per school was sampled, which made it impossible todisentangle classroom- and school-level variance. In two small countries, Cyprus and Malta,two classrooms were sampled in each school; a few other countries had only small numbers ofschools with more than one classroom. These differences in sampling design need to be takeninto account when interpreting the results of the multilevel analyses of ICCS data.Missing data treatment in ICCS multivariate analysesMultivariate analysis is more prone to missing data problems than are other forms of analysis.A larger number of cases tend to be excluded if the analysis uses only those records that havecomplete information for all variables. Generally, there are two possible sources of missingdata: (1) no questionnaire data for either the student or their school, and (2) missing data forindividual variables.To address the missing data issue, the small proportion of students without any studentquestionnaire data were excluded from the analyses and a “dummy variable adjustment” wasapplied for the remaining students (see Cohen & Cohen, 1975). Mean or median values wereassigned to students or schools with missing data, and dummy indicator variables (with 1indicating a missing value and 0 non-missing values) were added to the analysis.This treatment was applied to both the student and the school levels during the hierarchicallinear modeling. At the student level, the variables were as follows:• Years of expected further education (EXPEDYR);• Frequency of students’ use of media information on political and social issues (MEDINF);• Perception of openness in classroom discussions of political and social issues (OPDISC);• Reported parental interest in political and social issues (RPARINT);• Frequency of discussion of political and social issues with parents (PARDISC); and• Recent voting for class representative or school parliament (SCVOTE).Treatment of the missing school questionnaire data involved including just one indicatorvariable for completely missing school data. The variable indicated cases for which any of theabove variables had missing values. Only two countries had complete—or almost complete—school data.Table 13.4 shows the unstandardized regression coefficients for missing indicators for thecomplete hierarchical linear model (Model 4) reported in the international report (Schulz,Ainley, Fraillon, Kerr, & Losito, 2010, pp. 225ff.). Missing indicators in almost all countrieswere negatively associated with civic knowledge; however, given the substantial standard errors,the associations were often not significant. No consistent association was found between themissing-school-data indicators and civic knowledge.270ICCS 2009 technical report