ICCS 2009 Technical Report - IEA

Adapting and translating the ICCS assessment materialsIn this section of the survey activities questionnaire, NRCs provided information about the processof adapting and translating the ICCS achievement test, background questionnaires, andregional modules into national languages and contexts. Most NRCs identified national centerstaff as the individuals responsible for adapting and/or translating the survey instruments; injust over a third of all participating countries, the national centers also consulted, and thenoften worked with, outside specialists to conduct the translation. When NRCs were askedabout their experiences with the external-adaptation negotiation and translation-verificationprocesses organized by the ISC and the IEA Secretariat (respectively), a small number of themsaid that they did experience difficulties, mainly with respect to tight timeframes and reachingagreement on the adaptation of certain specialist concepts. Table 9.11 presents a summary ofthe information collected in this section of the questionnaire.Table 9.11: Numbers of NRC responses to the survey activities questionnaire: adapting and translating the ICCS assessmentmaterialsQuestion Adaptation Translation Not ApplicableWho adapted and/or translated the international version of the student test? 0(Select all that apply)• Own staff 30 19• Outside translator(s) 2 16• Outside reviewer(s) 5 4• Combination of the above 9 11Who adapted and/or translated the international version of the student, 0teacher,and principal questionnaires? (Select all that apply)• Own staff 31 18• Outside translator(s) 2 14• Outside reviewer(s) 4 4• Combination of the above 10 12Who adapted and/or translated the international version of the 3regional module instruments? (Select all that apply)• Own staff 27 17• Outside translator(s) 3 12• Outside reviewer(s) 1 3• Combination of the above 8 10Question Yes No Not AnsweredDid you have major problems regarding the process of externaladaptation verification and/or translation verification of the instruments?• Adaptation verification 6 31 1• Translation verification 6 30 2Assembling and printing the ICCS assessment materialsTable 9.12 shows that, as with the first two rounds of external verification, few countriesexperienced difficulties when completing the third step of the verification process—layoutverification (this was overseen by the ISC). Tight timelines posed the biggest challenge in twocountries. In general, NRCs reported no difficulties assembling the various survey instruments,although they detected, in a number of cases, printing errors, such as poor print quality, missingpages, wrong page order, and unintentional upside-down pages. In most cases, these errorswere minor and were resolved in a straightforward manner. All cases where errors could not becorrected were carefully documented.QUALITY ASSURANCE IN THE ICCS DATA COLLECTION115