ICCS 2009 Technical Report - IEA

Chapter 10:Data management and creation of theICCS international databaseMichael Jung and Falk BreseIntroductionThis chapter describes the procedures for checking ICCS data and database creation that wereimplemented by IEA’s Data Processing and Research Center (IEA DPC), the ICCS InternationalStudy Center (ISC) at the Australian Council for Educational Research (ACER), and thenational centers of the participating countries. The main purposes of these procedures were toensure the following:• All information in the database conformed to the internationally defined data structure;• The content of all codebooks and documentation appropriately reflected nationaladaptations to questionnaires; and• All variables used for international comparisons were comparable across countries.All institutions involved in this process applied control measures throughout it in order toassure the quality and accuracy of the ICCS data.Confirming the integrity of the ICCS international databaseThis process required close cooperation between the international and national institutionsinvolved in ICCS. Quality assurance comprised several steps. During the first step, staffat the IEA DPC involved in this process checked the data files provided by each country.They applied standard cleaning rules to verify the accuracy and consistency of the data,and documented any deviations from the international file structure. The IEA DPC sent anyqueries to national centers, the staff of which modified their data files where necessary. Afterall modifications had been applied, staff at the IEA DPC re-checked all datasets. This processof editing the data, checking the reports, and implementing corrections was repeated asmany times as necessary to ensure that all data were consistent within and comparable acrosscountries.After the national files had been checked, the IEA DPC provided national centers with nationalunivariate and reliability statistics as well as data almanacs containing international univariatestatistics and national item statistics. This material enabled national center staff to compare theirnational data against the international results.This step was one of the most important data-quality procedures used because it ensured theinternational comparability of the data. For example, a particular statistic that might haveseemed plausible within a national context could have appeared as an outlier when the nationalresults were compared against the international results. Staff at the IEA DPC reviewed all suchinstances and, when necessary, addressed it either by recoding the corresponding variables orremoving them from the international database.Once the national databases had been verified and formatted according to the international fileformat, national center staff sent their data to the ISC, which then produced and subsequentlyreviewed the basic item statistics. At the same time, the IEA DPC produced data files containinginformation on the participation of schools and students in each country’s sample. Staff at theIEA DPC then used this information, together with data provided by the NRC survey trackingforms 1 and the software designed to standardize operations and tasks, to calculate samplingweights, population coverage, and school, teacher, and student participation rates. 21 Survey tracking forms were used to record the sampling of schools, classes, teachers, and students (also see Chapter 8).2 Chapter 7 of this report provides details about the ICCS 2009 weighting procedures.121