I__. - International Military Testing Association

following: instructional experience, subject matter experience,
computer experience, and cognitive style. Some of this data was
gathered by direct questioning and was easily quantified (e.g.,
number of years of teaching experience). Cognitive style was
determined by questioning and by observation, and was not as
easily quantified. Some aspects of computer experience were
easily determined by questioning (e.g., number of computer
courses taken), but other aspects were not as straightforward
(e.cbI level of expertise with an operating system).
Initial process measures for this soft model included the
following: time spent on an authoring event, sequence chosen for
authoring events, number of revisions attempted and accomplished,
and purpose of revisions. Again some of these processes were
easy to measure and to quantify, but other processes were more
difficult to assess. For example, it was easy to measure how
long an author spent indicating the particular function of a
device that was part of the lesson content. However, determining
the purpose of a particular revision without interrupting the
integrity of the authoring process was more difficult. The only
way to accomplish this was to note that a revision had been made,
look at the revision, if its purpose was not obvious (e.g.,
correcting a misspelled word was an obvious revision), then the
author was asked after the session about the purpose of the
revision.
Initial output measures for this soft model included the
following: total time to produce the lesson module, total cost
to produce the lesson module, student achievement on tests,
retention, student motivation concerning the material, level of
interactivity of the lesson, instructor motivation to use the
authoring environment in the future, and peer review by other
instructional developers. Once again some of these measures are
direct and straightforwardly quantifiable (e.g., total
development time, student scores, etc.), while some are indirect
and more qualitative (e.g., instructor and student motivation).
The initial subject was observed completing a lesson module
to teaching the names, locations, and functions of 125 parts in
the T-37 cockpit. Subject's experience was determined in an
extensive interview prior to the study. Subject's motivation was
observed throughout the study. In addition, the subject was
queried midway through the study concerning his progress and
problems encountered. The subject also kept a diary of authoring
events, including problems encountered and general impressions.
Results
The relevant input measures of the subject were as follows:
1) Medium instructional experience, 2) High subject matter
experience, 3) Low computer experience, and 4) Reflective
cognitive Style with a self-directed locus of control. A formula
for connecting each of these factors with output measures is
currently being developed and will be tested in the second
_ _ ^ -..-.-.-
188