ICCS 2009 Technical Report - IEA

In the case of items with more than two (k) categories (as, for example, with Likert-type items),this model can be generalized to the partial credit model (Masters & Wright, 1997), 3 which takesthe form ofP xi (q) =m ixexpS (q n – d i + t ij )k=0kS exp S (q n – d i + t ij )h=0k=0x i = 0,1…,m i, (2)where P xi (q) denotes the probability of person n scoring x on item i, q n denotes the person’slatent trait, the item parameter d i gives the location of the item on the latent continuum, and t ijdenotes an additional step parameter.The weighted mean-square statistic (infit), which is a residual-based fit statistic, was used toassess item fit. Weighted infit statistics were reviewed for both item and step parameters, andACER Conquest software (Wu, Adams, Wilson, & Haldane, 2007) was used to estimate itemparameters and to analyze item fit.The international item parameters that were obtained came from the following calibrations.• Calibration of student item parameters: subsamples of 500 students randomly selected fromeach (weighted) national database for the 36 countries that met sample participationrequirements. The final calibration sample included data from 18,000 students.• Calibration of teacher item parameters: subsamples of 250 teachers randomly selected fromeach (weighted) national database for the 27 countries that met sample participationrequirements. The final calibration sample included data from 6,750 teachers.• Calibration of school item parameters: national school samples weighted to have the sameweight (set to values of 100 regardless of sample size) for each country that met sampleparticipation requirements. The final calibration sample included data from all schoolprincipals.After the international item parameter from the calibration sample had been estimated,weighted likelihood estimation was used to obtain individual student scores. Weightedlikelihood estimations can be computed by minimizing the equationSi∈Ωr x +J n2I nk– Sj=1m ixexp(Sq n – d i + t ij )j=0kS exp S (q n – d i + t ij )h=0k=0= 0(3)for each case n, where r x is the sum score obtained from a set of k items with j categories. Thiscan be achieved by applying the Newton-Raphson method. The term J n /2I n (with I n beingthe information function for student n and J n being its derivative with respect to q) is used as aweight function to account for the bias inherent in maximum likelihood estimation (see Warm,1989). ACER ConQuest software made it possible to pre-calibrate item parameters in order toderive scale scores.The weighted likelihood estimates were transformed to an international metric with an ICCSaverage of 50 and a standard deviation of 10 for equally weighted datasets from the 36countries that met sample participation requirements. The following formula was applied inorder to achieve the transformation:qq’ n = 50+10n – – q ICCS,σ q (ICCS)3 An alternative is the rating scale model (RSM), which has the same step parameters for all items in a scale (see Andersen,1997).162ICCS 2009 technical report