ICCS 2009 Technical Report - IEA

Table 2.6: Field trial item mapping to assessment frameworkContent domainPercentage of all itemsCivic society and systems 40Civic principles 30Civic participation 20Civic identities 10Total 100Cognitive domainKnowing 30Reasoning and analyzing 70Total 100Because the ICCS framework was broader than the framework for CIVED, most of the CIVEDitems were mapped to the content domain civic society and systems and the cognitive domainknowing. The newly developed ICCS items were written to complement the framework coverageof the CIVED items and consequently they refer mainly to the content domains of civicprinciples, civic participation, and civic identities, and the cognitive domain reasoning and analyzing.The decision to have relatively low proportions of items addressing the content domains civicparticipation and civic identities was made because little of this kind of content could reasonablybe asked of students in this age group. These two content domains were given a stronger focusin the student questionnaire. Tables 2.5 and 2.6 show that the originally anticipated and finalproportions of the field trial items provided good coverage of the assessment framework’scontent domains.Main surveySelection of itemsEvaluation of the measurement properties of the field trial test items was based on thedata collected from students in 32 participating countries. The analysis procedures used toreview measurement properties are described in Chapter 11. The items with unsatisfactorymeasurement properties were further reviewed to determine whether they could be revised orwhether they needed to be deleted from the item set. Minor revisions were made to 25 items.Items were only modified when there was clear evidence that the revision would improve theirmeasurement properties. Eighteen items were removed from the ICCS test item pool, leaving 80items for inclusion in the main survey.Test design and contentThe main survey test consisted of 80 items, including 17 items that were secure items from theIEA CIVED study. Table 2.7 shows the composition of the main study test instrument by itemtype and item origin.Based on experience with the field trial, the test development team decided to reduce thetesting time from 60 to 45 minutes. This decision meant that the number of items within eachcluster had to be reduced and the number of booklets increased.28ICCS 2009 technical report