ICCS 2009 Technical Report - IEA

• Not reached (used only in the achievement files): this code indicated those items that students didnot reach because of a lack of time. 4Data productsData products sent to national centers by the IEA DPC and the ISC included both itemstatistics and data files.Item statisticsISC staff produced and then sent a set of item statistics to each national center for review.Each set contained weighted summary statistics for the participating country on each variableincluded in the survey instruments. The ISC also used these datasets during their data reviews.In addition, IEA DPC staff produced a set of preliminary scoring reliability statistics for eachnational dataset. This contained summary statistics at the item level on the percent of agreementbetween scorers.Versions of the national data filesBuilding the international database was an iterative process. On completion of each majordata-processing step, the IEA DPC sent a new version of data files to the national centers sothat staff could review their data and run their own separate checks to validate the new data-fileversions. This process meant that national centers received several versions of their data, andtheir data only, before the international database was published. IEA DPC staff sent the first ofthese versions as soon as they considered the data “clean” with respect to identification codesand linkage issues.These first sets of files contained nationally standardized achievement scores that the IEA DPCcalculated by means of Rasch scaling. They were also accompanied by documentation thatincluded a list of the cleaning checks and corrections made to the data so that the nationalcenters could review the cleaning process. The IEA DPC sent national centers another versionof the data files together with data almanacs once the weights and international achievementscores were available. This step did not take place until all tables and figures contained in theICCS international reports had been verified and final updates to the data files implemented.This approach enabled national center staff to replicate the results presented in the internationalreports.The ICCS international databaseThe ICCS international database incorporated all national data files from participatingcountries. The data processing that the IEA DPC conducted ensured that:• Information coded in each variable was internationally comparable;• National adaptations were reflected appropriately in all variables;• Questions that were not internationally comparable had been removed from the database;• All entries in the database could be linked to the appropriate respondent—student, teacher,or principal;• Only those records adjudicated as participating remained in the international databasefiles; and• Sampling weights and student achievement scores were available for internationalcomparisons.More information about the ICCS international database is provided in the ICCS User Guide forthe International Database (Brese et al., 2011).4 “Not reached” codes were derived as follows: an item received this coding if the student concerned did not respond to anyof the items following it (i.e., did not continue on to the end of the test) and/or if he or she did not respond to the itempreceding it.DATA MANAGEMENT AND CREATION OF THE ICCS INTERNATIONAL DATABASE127