I__. - International Military Testing Association

iteration of the evaluation study.
The relevant process measures were as fOllOWS: 1) 4.75
hours in introductory exercises, 2) 14.25 hours in on-line
authoring, 3) 11.83 hours in off-line design and planning, 4) 10
groupings, nested 3 levels deep, with a total of 21 lesson
modules, top level module completed first, teaching 125 parts, 5)
20 picture files identified and utilized, with minor revisions
requested for 4, 6) Approximately two minor revisions per module,
7) Approximately 5 minutes of debugging per individual module,
and 8) Complete linkage of all modules into a course module in 20
minutes. This data was collected by observation. The software
has since been modified to collect and record this data
automatically (Canfield & Spector, 1990).
The relevant output measures were as follows: 1) 30.83
hours in total development time (graphics were produced by
support personnel and graphic production time is not included),
2) 3 plus hours expected for student instructional time, 3) cost
data not available, 3) student scores and motivation not
available, 4) medium level of interactivity, 5) high instructor
motivation (wants to be included in follow-on studies, and 6)
acceptable.guality of courseware (will be administered to cadets
in lieu of current instruction).
The subject's diary and responses to interview questions
indicated a sustained high level of motivation and satisfaction
with the authoring tool in spite of known deficiencies
(occasional mouse failures). The subject experimented with
default instructional parameters during the exercises but rarely
changed the defaults for the instruction he developed. More
specifically, the subject chose timed presentations for the
student practice interaction rather than learner control. The
subject also modified the default testing parameters to reflect 3
samples per item instead of 2 and a criterion level of 75%
instead of 90%. In addition, the subject altered allowable
interactions per individual lesson as appropriate, which
reflected complete understanding of the transaction shell
environment.
Conclusion
This initial study prompted the addition of automatic data
collection for both instructors and students to the transaction
shell software. The general results indicate a high level of
acceptability and productivity using transaction shells to author
courseware. Assessment of the quality of the CBI produced has
yet to be completed, although initial data collection on student
performance is underway.
Initial indications are that students require in excess of
3 hours to complete the course module. This means that the
subject's development time to instruction time ratio using this
tool was approximately 1O:l. Using traditional authoring tools
189
_