awej 5 no.4 full issue 2014

AWEJ Volume.5 Number.4, 2014Fair, Reliable, Valid: Developing a Grammar Test UtilizingEfeotorTableMike4.OrrThe items with the lowest correlationThe substantive aspect of validity ―deals with finding empirical evidence to assure that testtakersare actually engaged with the domain processes provided by the test items or tasks‖(Baghaei & Amrahi, 2011, p.1052). One way of evaluating this is by doing a multiple choicedistracter analysis, looking at the P-values, which would show ―the degree to which theresponses to the distracters are consistent with the intended cognitive processes around which thedistracters were developed‖ (Wolfe & Smith, 2007, p. 209). However, as was previouslymentioned, the distracters are mainly homogenous, and so no conclusions relating to theconstruct map were drawn from the distracters examinees chose. Another indicator of thesubstantive aspect of validity is person fit statistics. Person fit statistics concern ―the extent towhich a person's pattern of responses to the items correspond to that predicted by the model‖(Smith, 2001, p. 296). Table 5 shows the highest and lowest INFIT and OUTFIT mean-squarestatistics.Table 5.The highest and lowest INFIT and OUTFIT mean-square and Z-StandardizedstatisticsArab World English JournalISSN: 2229-9327www.awej.org216