Technical Report - International Military Testing Association

.
.
are to be developed. Discus6 with the subject-matter experts the posl;ible
variation8 of the procedure 80, that they have clearly in mind what is
deefred and how to provide it. (See Annex 3 for a list of criteria for
m iteIII8.)
4. Develop an experimental form of the PCT, coixsidering the "item
criteria” lieted in Annex 3 and including all of the Items which are judged
to be promising.
5. Administer this experfmental form to a group of men working in
the occupational specialty, a8 close as fe feasible to the date on which
the specialty is being evaluated by the regular instruments. In the
rample used, all levels of proficiency should be reprerented. At least
two, and preferably three or more-observers of each men should apply the
experimental PCT. At the 8ame time the be8t posrible externel criterion1
data should be obtained,
6. Analyze the item8 and the possible ways of scoring each against
the total score on the experimental form of the PCT, against the written
test, and against the external criterion. Obtain correlation8 also with
all of the regular evaluation instruments. The consistency with which
each ttw can be used will be analyzed. Techniques similar to thorc, used1
fa refining biographical information blank8 and to regular item analyser
should be applied. The desirability of assigning other than unitary
weights, to the individu81 item8 In the check teat, should be determined
empirically. The most valid and relipble scoring points for each item
should lfkewise be determined empirically.
7. Coneiderfng all the available data and the pertinent general
measurement criteria, develop the revised form of the Performance Check
Teat. If extensive modification of a eignlficant proportion of the experimental
items has not been neceeeary, a useful estimate of the reliability
of the total score on the revised Performance Check Test @CT)
can be obtained at this time. The data also can be analyzed in relation
to minimum acceptable proficiency 8nd promotion. Optimal Weight6 of all
the evalucltion instruments naturally should be determined for the experimental
tryout population snd appropriate personnel action8 determined
for critical “cut score8.”
Por jobs in which it ir pO8Sible to obtain an adequate number of PCT
items, each of which can be consistently observed in a performance as
meeting or not meeting the deecribed limits--e.g. for which the observer
can reliably say,“This man can, (0. cannot) do this thing as specified,”
and when this crucial statistical .-nalysis la made, the effectivenees
of this Performance Check Test technique should be assured, The percentage
95
.