Technical Report - International Military Testing Association

,
s
.
. ---_.
. . .
.
.
- -.--. .-. ._- .*__ ____-.._ -.-..-- -- --.. _.-
“-. _-- ._-__.- .- ---
Ideally, the items should NOT require the observer to evaluate the
level at which the individual performs the skill covered by the item.
In ‘other words, it should not be necessary to make qualitative judgment
or inferences in deciding uhether or not a given exadntc performs the
skill in the manner defined by the item.
In btmary, the most important distinguishing characteristics of a
Performance Check Test are that (1) the performances can readily be observed,
(2) they can be consistently judged as being present to the degree described,
or absent, and (3) they are selected by appropriate item analysis and
validation techniques.
The Performance Check Test might be considered as consisting of a
special type of rating scales. Honever, the PCT is not a usual type of
rating scale. It is a li.st of things which an observer can easily determine
that a man can or cannot do. These should be weighted on the
basis of experimental findings which will indicate each Item’s Importance
as a component of the exsminee’s effectiveness cr job proficiency. The
quantitative expression of the level of skill in the manual manipulative
aspects of the job vould be obtained mensly by adding up the vei&tc
assigned to the items checked for the individual evaluated on the
Performance Check Test. A total score reliably and validly reflecting
the exsminee’s skill is obtained. It covers discriminating motor aspects
, of the job. When appropriately weighted and combined with the written
test and other job,proficiency evaluations, a significant increase and
in some jobs a considerable increase in validfty should result. Thus
additional inroads into currently unmeasured variance in job proficiency
should result.
SlJHKARY OF DEVELOPMENTAL PROCEDURE
1. ldenttfy the jobs for which current evaluation instruments are
least valid in which motor skills or their product are important
discriminators of d!.fferences in levels of job amstery, and in which
the tasks involved most adequately meet the criteria, (See Annex 1).
2. Analyze all of the pertinent information available or readily
attainable and prepare a preliminary list of items to serve as an
illustration for the subject-matter experts engaged in developing a long
experimental list of potential PCT items. The “critical incident” approach
developed by Dr. John C. Flanagan would be quite helpful. The “essay
approach” proposed by Hr. Roberts, an USARES Supervisory Research Psychologi at,
and his staff should be another helpful source of items. (See Annex 2 for
their materials.)
3. Convey these examples together with guidelines and appropriate
accessory materials to the appropriate experts on the occupational
specialty involved in the organizations at which such experimental items
94
.
._