ICCS 2009 Technical Report - IEA

Countries were expected to use the same translation for the ICCS items as was used in theprevious assessment. In some cases, however, national centers considered modifications tothe translations from CIVED absolutely necessary. In these cases, the changes were carefullydocumented and referenced during the data analysis. If the changes seemed to have altered theperformance of any item, this item was not included in the scaling of link items to measurechanges in civic content knowledge since 1999 for that particular participating country.International verification of the instrumentsIn addition to the internal review of translations carried out by each national center, all surveyinstruments went through a rigorous three-part international verification process: (i) adaptationsnegotiation, (ii) translation verification, and (iii) layout verification. An independent review ofthe translation verification record was also conducted by international quality-control monitors,as part of the ICCS quality assurance program.Documentation in the national adaptations formWhen translating and adapting the international version of the instruments for national use,national centers needed to make certain changes, selections, and adaptations to the surveyinstruments. In doing so, they were required to keep in mind that the objective of the study wasto create an international database containing comparable data from all participating countrieswith complete documentation. Consequently, any change that was made had to be recordedelectronically on the NAF. This form was used not only for documentation purposes, but werealso when national data were added to the international database.The NAF in Microsoft ® Excel format consisted of several worksheets for the cognitive test,student questionnaire, teacher questionnaire, school questionnaire, regional test/questionnaire,and additional country documentation (language information, version number, and inclusionstatus of international optional questions). National centers were required to complete aNAF for each survey language used in their country. National centers were also requested todocument whether they intended to include any of the international options or any nationalitems and categories (together with a description of their content in both the national languageand English). For those countries measuring overtime comparisons in achievement, adaptationsfor existing CIVED items had to be documented in the NAF, in the same way as for the othernational adaptations.The NAFs were completed and reviewed at various stages of the verification process. Detailedinstructions on how to work with the NAFs and how to adapt the data-entry software weretherefore provided to national center staff at data management seminars preceding the field trialand main survey.Adaptations negotiationNRCs were required to consult with ISC staff when reviewing all proposed nationaladaptations. In particular, they were strongly encouraged to discuss any adaptation that mightresult in a serious deviation from the items in the international instruments.National centers began completing the NAF (Version I) after reviewing the international versionof the survey instruments. They submitted the NAF to the ISC and consulted with them onthe form’s contents. Following the review process, the ISC provided the national centers withfeedback on their adaptations and, where appropriate, suggested improvements.National centers were requested to take the recommendations into account and update theforms accordingly so that these updated forms (Version II) could be used during the translationverification process to evaluate the quality and accuracy of the translations.TRANSLATION AND NATIONAL ADAPTATIONS OF ICCS 2009 INSTRUMENTS55