ICCS 2009 Technical Report - IEA

Table 2.7: Composition of the main study test instrument by item type and originItem format New items CIVED items Total Total score Percentage ofpoints of score pointsMultiple choice 57 17 74 74 86Constructed response 6 – 6 12 14Total 63 17 80 86 100For the main survey, test items were allocated to seven clusters that were assembled into a fullybalanced rotated test design comprising seven test booklets, each with a testing time of 45minutes. One cluster of items comprised the set of secure CIVED items. Table 2.8 shows thecluster composition and Table 2.9 shows the rotated cluster test design used in the main survey.The CIVED trend-item cluster is shaded in Table 2.9.Table 2.8: Main study cluster compositionClusterC1C2C3C4C5C6C7 (CIVED)Items10 (9 MCQ, 1 CRQ)10 (9 MCQ, 1 CRQ)10 (9 MCQ, 1 CRQ)11 (10 MCQ, 1 CRQ)11 (10 MCQ, 1 CRQ)11 (10 MCQ, 1 CRQ)17 MCQNote: MCQ = multiple-choice question; CR Q = constructed-response question.Table 2.9: Main study test booklet designPositionBooklet 1 2 31 C1 C2 C42 C2 C3 C53 C3 C4 C64 C4 C5 C75 C5 C6 C16 C6 C7 C27 C7 C1 C3Note: CIVED link cluster shaded in grey.Mapping to frameworkTable 2.10 shows the mapping of the main survey test items to the ICCS assessmentframework.A comparison of Table 2.10 and Table 2.5 reveals that the final test instrument provided agood coverage of the assessment framework and matched the original test-development plan.However, it needs to be acknowledged that, when compared to the originally anticipatedcoverage, the main survey test included a somewhat higher proportion of items relating to thecognitive domain reasoning and analyzing.ICCS test development29