28th International Congress of Psychology August 8 ... - U-netSURF

3116 ORAL
Educational psychology
Chair: Hong Jiao, USA
3116.1 The effect of missing data of rating design on parameter estimations using the
many-facet Rasch model, Shudong Wang, Michael Young, Holly Zhang, Harcourt Assessment,
Inc., USA
The Many-facet Rasch model (MFRM) has been used to calibrate and scale performance
assessments that involve subjective judgments. The trade-off using an unbalanced rating design
has been to sacrifice the accuracy of parameter estimates for the low cost and relative ease of
implementing these designs. The purpose of this Monte Carlo study was to evaluate the effects of
missing data on person, item, and rater facet parameter recovery using MFRM under a variety
balanced and unbalanced rating designs. This study shows that the missingness rate can be
considered an indicator of the quality of parameter estimation using the MFRM.
3116.2 The effects of random sampling on the Mantel Statistics, Hong Jiao, Shudong Wang,
Harcourt Assessment, Inc., USA
This study is to investigate the effects of random sampling on the Mantel Statistics in detecting
differential item functioning (DIF) in polytomous items. Different samples from the same
population will be selected and the magnitude, the direction, and the uniformity of DIF in
polytomous items are compared to examine the consistency in detecting DIF based on the Mantel
statistics in polytomous items across different groups of examinees. Both real data and simulated
data will be used in this study. Several factors are manipulated. They are sample size, population
distribution, DIF magnitude and direction.
3116.3 The consistency of using different matching variables in detecting differential item
functioning, Hong Jiao, Shudong Wang, Harcourt Assessment, Inc., USA
This study investigates whether using different matching variables will result in differences in
detecting Differential Item Functioning (DIF) in both dichotomous and polytomous items. Four
types of scores, the total raw score, stanine, scale score, and performance level are used as
matching variables respectively. Both real data and simulation data are used in this study. The
simulation study simulates different magnitude of DIF. Four DIF indexes are applied to detect DIF,
namely, Mantel statistic, Mantel-Haenszel statistic, unsigned/absolute standardized mean
difference, and signed standardized mean difference.
3116.4 Comparison of computerized adaptive test and computerized classification test in
making classification decisions, Hong Jiao, Shudong Wang, Allen Lau, Harcourt Assessment,
Inc., USA
The purpose of this study is to investigate whether the improvement in the ability estimation using
weighted likelihood estimation (WLE; Warm, 1989) in computerized adaptive test (CAT)
enhances the classification accuracy compared to computerized classification test (CCT) utilizing
sequential probability ratio test (SPRT; Wald, 1947) when both procedures are applied to make
dichotomous classification decisions. This study helps to determine whether the improvement in
the ability estimation using WLE will increase the test accuracy and efficiency in WLE-based CAT
745