ICCS 2009 Technical Report - IEA

Identification variable (ID) cleaningEach record in a data file needs to have a unique identification number. The existence ofrecords with duplicate ID numbers in a file implies an error of some kind. If two records in anICCS database shared the same ID number and contained exactly the same data, the IEA DPCdeleted one of the records and kept the other one in the database. If both records containeddifferent data and IEA DPC staff found it impossible to identify which record contained the“true data,” they removed both records from the database. The IEA DPC tried to keep suchlosses to a minimum; actual deletions were rare.Although the ID cleaning covered all data from all instruments, it focused mainly on thestudent questionnaire file, which contained most of the critical ID variables. In additionto checking the unique student ID number, IEA DPC staff also needed to check variablespertaining to student participation and exclusion status, as well as students’ dates of birth anddates of testing in order to calculate student age at the time of testing. The student trackingforms provided an important tool in relation to resolving anomalies in the database. The IEADPC conducted all cleaning procedures in close cooperation with the national centers. Afternational center staff had cleaned the identification variables, they passed the clean databaseswith information about student participation and exclusion on to the IEA DPC samplingunit, which used this information to calculate students’ participation rates, exclusion rates, andstudent sampling weights (see Chapter 7 for details).Linkage checkBecause, in ICCS, data about students, their schools, and teachers appeared in a number ofdifferent files, it was crucial to correctly link these records so as to provide meaningful datafor analysis and reporting. Linkage was implemented through a hierarchical ID numberingsystem that included a school, class, and student component, 3 cross-checked against thesurvey tracking forms. Student ID values in the achievement file and student questionnaire filehad to be matched correctly, as did those in the reliability scoring file and the achievementfile. In addition, it was important to ensure that teacher and student records linked to theircorresponding schools.Resolving inconsistencies in questionnaire dataThe amount of inconsistent and implausible responses in questionnaire data files variedconsiderably among countries. However, none of the national datasets was completely free ofinconsistent responses. The IEA DPC determined the treatment of inconsistent responses on aquestion-by-question basis, using all available documentation to make an informed decision.IEA DPC staff also checked all questionnaire data for consistency across the responses given.For example, Question 20 in the school questionnaire asked for the total school enrolment(number of students) in all grades, while Question 21 asked for the enrolment in the targetgrade only. Clearly, the number given as a response to Question 21 could not possiblyexceed the number provided by school principals in Question 20. The IEA DPC flaggedinconsistencies of this kind and then asked the national centers to review these issues. IEA DPCstaff recoded as “invalid” those cases that could not be corrected or where the data providedmade no sense.3 The ID number of a higher level is included in the ID number of a lower sampling level. The class ID includes the schoolID, and the student ID includes the class ID (e.g., student 10120523 may be described as student 23 of class 05 in school1012).DATA MANAGEMENT AND CREATION OF THE ICCS INTERNATIONAL DATABASE125