I__. - International Military Testing Association

Table 4
Additive Questionnaire Instrument for Estimating Trials-to-Mastery
IF AN AVERAGE STUDENT REQUIRES *N* TRIALS TO LEARN TO
MASTERY, HOW MANY TRIALS WILL A . . . FAST LEARNER REQUIRE?
. . . SLOW LEARNER REQUIRE?
IF AN AVERAGE INSTRUCTOR REQUIRES *N* TRIALS TO TRAIN
STUDENTS, HOW MANY TRIALS WILL . . . AN EASY INSTRUCTOR NEED?
.., A TOUGH INSTRUCTOR NEED?
IF *N* TRIALS ARE NEEDED FOR AVERAGE TASKS, HOW MANY
TRIALS WOULD... . . . AN EASY TASK REQUIRE?
. . . A TOUGH TASK REQUIRE?
VALIDATION AND APPLICATION
The model was validated in the helicopter community using a concurrent
validation design. Criterion data for the simulation were collected during
an experimental evaluation of Device 2F64C, an SH-3 simulator located at the
Naval Air Station, Jacksonville, Florida. Trials-to-mastery obtained from
the simulation model were compared with the trials-to-mastery obtained from
the field experiment (Evans, Scott & Pfeiffer, 1984).
SUBJECTS AND PROCEDURE
Thirteen flight instructors currently involved in training pilots in
Device 2F64C were asked to estimate trials-to-mastery by two different
methods. The subjects, one at a time, made their estimates at a computer
terminal. One half-hour per subject was required to complete both the
additive and interactive rating tasks.
VARIABLES
Four independent variables (shown following ) were included in the
validation desi n: (XI) device feature, (X2) student ability, (X3) task
difficulty and PX4)
instructor leniency. All combinations of two levels for
Xl and three levels for X2, X3, and X4 produced 54 data points for a
re ression analysis against estimated trials-to-mastery. Trials-to-mastery
(Yy in the
aircraft was the dependent variable.
EVALUATION DESIW SE#SITIVIM
The usual purpose of a device feature evaluation is to extract the
variance due to the device features, e.g., visual and motion vs. motion
onlY.The modeled data can also be used to do a power analysis of the onetrial
difference (actually A = 1.04) between device features. Power
analysis provdes an estimate of the sample sizes needed to demonstrate that
this one-trial difference (experimental mean = 4.61, SD = 1.83: control
mean = 5.65, SD = 2.07) is reliable (pfeiffer, Evans & Ford, 1985).
196