Modeling and Multivariate Methods - SAS

Chapter 22 Scoring Tests Using Item Response Theory 529
Launching the Platform
Figure 22.7 Item Analysis Launch Dialog
Y, Test Items are the questions from the test instrument.
Freq
optionally specifies a variable used to specify the number of times each response pattern appears.
By performs a separate analysis for each level of the specified variable.
Specify the desired model (1PL, 2PL, or 3PL) by selecting it from the Model drop-down menu.
For this example, specify all fourteen continuous questions (Q1, Q2,..., Q14) as Y, Test Items and click OK.
This accepts the default 2PL model.
Special Note on 3PL Models
If you select the 3PL model, a dialog pops up asking for a penalty for the c parameters (thresholds). This is
not asking for the threshold itself. The penalty it requests is similar to the type of penalty parameter that you
would see in ridge regression, or in neural networks.
The penalty is on the sample variance of the estimated thresholds, so that large values of the penalty force
the estimated thresholds’ values to be closer together. This has the effect of speeding up the computations,
and reducing the variability of the threshold (at the expense of some bias).
In cases where the items are questions on a multiple choice test where there are the same number of possible
responses for each question, there is often reason to believe (a priori) that the threshold parameters would be
similar across items. For example, if you are analyzing the results of a 20-question multiple choice test where
each question had four possible responses, it is reasonable to believe that the guessing, or threshold,
parameters would all be near 0.25. So, in some cases, applying a penalty like this has some “physical
intuition” to support it, in addition to its computational advantages.