ICCS 2009 Technical Report - IEA

Test coverage and item dimensionalityWhen measuring cognitive abilities, it is important to use test items that cover the differentlevels of achievement found in the target population. Figure 11.1 shows the distributionof cognitive abilities among ICCS students (for the representative sample used for the finalcalibration) and the location of items (at rp = 0.5).The range of item difficulties generally matched the abilities found in the student population,but the average item difficulties were somewhat lower than the average student abilities.Overall, the test items were better at targeting students in the lower than in the higher civicknowledge ranges. However, the nature of this targeting varied across countries according tothe distribution of student achievement within each country.Multidimensional item response models were used to assess the dimensionality of items. Twoof the possible item dimensions that were explored are based on the structure of the cognitivedomains described in the ICCS assessment framework (Schulz et al., 2008).Figure 11.1: Mapping of student abilities and item difficultiesNote: Calibration results for the international calibration sample (adjudicated items only).130ICCS 2009 technical report