2003 IMTA Proceedings - International Military Testing Association

472
ITEM SELECTION SCREENS
First Screen
Attachment B
1. Endorsement rates (er) equal to or greater than .1 for all subgroups
2. Item-to-scale correlation of .4 and above
3. Item-to-scale correlation larger than the same item adjacent scales
Second Screen
Categorize and sort items into three categories.
Best Items
1. Absolute difference of endorsement rates for all subgroups no greater than .15
2. Item-to-scale correlation of .5 and above
3. Strong hexagonal correlation pattern
Good Items
1. Absolute difference of endorsement rates for all subgroups no greater than .18
2. Item-to-scale correlation of .4 to .49
3. Correlation pattern approximates the hexagon
Marginal Items
The item meets at least two of the three Good item criteria.
Third Screen
1. Evaluate the coverage of each scale domain by the Best and Good performing items.
Identify gaps and redundancy.
2. Evaluate the balance of each scale from the perspective of each subgroup to ensure
balance.
3. Consider the possibility of using Marginal items.
4. Consider each Training and Education and Occupations items to see whether they can be
rewritten as an Activity item.
Fourth Screen
1. Compare individual judgments and come to consensus.
2. Identify gaps and weaknesses in domain coverage.
3. Conduct analyses of new interim scales.
4. Estimate numbers of new items needed by scale.
45 th Annual Conference of the International Military Testing Association
Pensacola, Florida, 3-6 November 2003