ICCS 2009 Technical Report - IEA

where q’ n are the scores in the international metric, q n are the original weighted likelihoodestimates in logits, and q – ICCS is the international mean of logit scores with equally weightedcountry subsamples. σ q(ICCS) is the corresponding international standard deviation of the originalweighted likelihood estimates. Table D.4 in Appendix D presents the means and standarddeviations used to transform the original scale scores for the international student, teacher,school, and regional (Asia, Europe, Latin America) questionnaires into the international metric.Describing questionnaire scale indicesFor the questionnaire scales, the weighted likelihood estimates (logits) for the latent dimensionswere transformed to scales with an ICCS average of 50 and a standard deviation of 10 (withequally weighted samples). While these scores could be interpreted by comparing individualscores or group average scores with the ICCS average, the individual scores do not revealanything about the actual item responses. Also, it would have been impossible to determinefrom the scale score values the extent to which respondents endorsed the items used to measurethe latent variable. The scaling model that was used to derive individual scores made it possibleto develop descriptions of these scales because scale scores could be mapped to (expected) itemresponses. 4It is possible to describe item characteristics by using the parameters of the partial credit modelto provide an estimate for each category of its probability of being chosen as a minimumrelative to all other categories. This process is equivalent to computing the odds of scoringhigher than a particular category.Figure 12.1 presents the results of plotting these cumulative probabilities against scale scores fora fictitious item. The three vertical lines denote those points on the latent continuum where itbecomes more likely to score > 0, > 1, or > 2. These locations Γ k are Thurstonian thresholds thatcan be obtained through an iterative procedure that calculates summed probabilities for eachcategory at each (decimal) point on the latent variable.Summed probabilities are not identical to expected item scores and have to be understoodin terms of the probability of scoring at least a particular category. Other ways of describingitem characteristics based on the partial credit model are item characteristic curves, which involveplotting the individual category probabilities and the expected item score curves (for a detaileddescription, see Masters & Wright, 1997).Thurstonian thresholds can be used to indicate for each item category those points on a scaleat which respondents have a 0.5 probability of scoring this category or higher. For example,in the case of Likert-type items with the categories strongly disagree (SD), disagree (D), agree (A),and strongly agree (SA), we can determine at what point of a scale a respondent has a 50 percentlikelihood of agreeing with the item.The item-by-score maps included in ICCS reports predict the minimum coded score (e.g.,0 = “strongly disagree,” 1 = “disagree,” 2 = “agree,” and 3 = “strongly agree”) a respondentwould obtain on a Likert-type item. For example, we could predict that students with acertain scale score would have a 50 percent probability of agreeing (or strongly agreeing)with a particular item (see the example item-by-score map in Figure 12.2). For each item, itis thus possible to determine Thurstonian thresholds, the points at which a minimum itemscore becomes more likely than any lower score to occur and which determine the boundariesbetween item categories on the item-by-score map.4 A similar approach was used in the IEA CIVED survey (see Schulz, 2004).SCALING PROCEDURES FOR ICCS questionnaire ITEMS163