I__. - International Military Testing Association

The performance tests utilized product scoring wherever a product was produced
as a complete or partial result of the performance. Where feasible, product
scoring is desirable because, correctly administered, it can enhance
reliability. The nine tests that utilized at least partial product scoring
are identified as such in Table 1.
In addition to the scoresheets, developers prepared equipment setup
instructions, instructions to the examinees, and scoring instructions. The
entire test was designed to be administered at a single station using either
an actual or a simulated ship's radio shack. Although the 14 tests were
independent, they were operationally interconnected so they fit logically and
sequentially into the test situation and location.
Written Tests
Written tests were developed that corresponded to the 22 tasks covered
in the hands-on tests. Three features characterized these tests:
. The tests were performance or performance-based. Items were
based on either performing the same steps required in the
hands-on test or in answering a question of how a step is done.
� The tests were founded on performance errors. To insure items
were performance oriented, the causes of error in performance
were identified. Error was identified as having four origins:
The Radioman did not know where to perform (location), did not
know when to perform (sequence), did not know what the product
of correct performance was (recognition), or did not know how
to perform (technique).
� The tests provided likely behavioral alternatives. Incorrect
alternatives were based on likely errors that were possible and
do occur on the job. Incorrect alternatives also had to be
wrong, not merely less desirable than the correct alternative.
The development result was an 87 item test in a multiple choice format
that was organized into 11 topical, functional task areas that generally
corresponded to the 14 hands-on test areas. (Several of the hands-on test
areas that needed to be treated separately for administrative and equipment
set-up requirements were combined for the written test, and one written test
task area did not survive validation.) These 11 written test areas were
organized so they could be administered and analyzed independently.
General Knowledae Test
The third area of RM testing was a written general knowledge test. Like
the written performance test, this was a multiple choice test and was based on
the same tasks that generated the hands-on tests. The difference between the
two written tests was that the written performance test was specifically
designed to measure performance while the general knowledge test measured the
application of knowledge to the task subject--which may not necessarily
reflect performance. For example, the written performance test might describe
a situation and ask what EMCOM condition should be imposed under those
circumstances; the general knowledge test might ask what EMCOM is.
530