ICCS 2009 Technical Report - IEA

Table 11.14: Item parameters, item fit, and item-total correlations from tentative calibration of Europeanregional test itemsItem Calibrated item Weighted item fit Item-totalparametercorrelationES2T01A -3.51 (0.05) 0.99 0.15ES2T01B -1.80 (0.03) 0.98 0.19ES2T01C -0.68 (0.02) 1.06 0.01ES2T02 -2.74 (0.04) 0.99 0.15ES2T03 -0.29 (0.02) 0.99 0.20ES2T04 0.41 (0.02) 1.01 0.15ES2T05 -0.70 (0.02) 0.96 0.29ES2T06 0.64 (0.02) 1.04 0.05ES2T07 0.27 (0.02) 0.99 0.20ES2T08 -0.30 (0.02) 0.97 0.25ES2T09A -0.61 (0.02) 1.00 0.17ES2T09B -2.23 (0.03) 0.97 0.22ES2T09C -1.87 (0.03) 0.98 0.21ES2T09D -0.88 (0.02) 1.01 0.13ES2T09E -0.09 (0.02) 1.04 0.08ES2T10 0.92 (0.02) 1.04 0.04ES2T11A -0.86 (0.02) 0.96 0.27ES2T11B 0.07 (0.02) 1.02 0.11ES2T11C -0.74 (0.02) 1.02 0.11ES2T12 -0.68 (0.02) 0.98 0.23Given the short test length, the unsatisfactory scaling properties of many of the European testitems, and the rather low overall reliability of the European test, a decision was made to reportonly item percentages in the final report (see Kerr, Sturman, Schulz, & Burge, 2010).The Latin American ICCS civic knowledge test consisted of 16 multiple-choice items thatfocused on specific aspects of knowledge relevant to the Latin American region. Althoughthe test items were specifically developed for students in Latin American countries, and thusaddressed aspects not relevant in other geographical regions of the world, the regional testitems were designed to measure the same content and cognitive dimensions as those in theinternational test. Responses to these items therefore reflected the same latent construct of civicknowledge, which made it possible to calibrate the regional items on the ICCS internationalcivic knowledge scale. That, in turn, meant that the item parameters could be compared withthe ICCS international civic knowledge scale and reported against the international proficiencylevels (see Schulz, Ainley, Friedman, & Lietz, 2011).Table 11.16 shows the calibration results for the 16 Latin American test items when scaledalongside the international test items with fixed item parameters. All test items had satisfactoryitem statistics.Table 11.17 records the reliabilities (Cronbach’s alpha) for the Latin American regional tests.The internal consistency of this item set was 0.72 for the pooled ICCS sample; nationalreliabilities ranged from 0.64 (in the Dominican Republic) to 0.76 (in Chile). The testcomponent thus had satisfactory internal consistency for the pooled sample and in three of theparticipating Latin American countries.SCALING PROCEDURES FOR ICCS TEST ITEMS153