I__. - International Military Testing Association

and various other sources, we became familiar with the training
program as it existed prior to the Model School Project, the
objectives of the project and changes made to the training program
as a result of it, and other occurrences which, although they may
have been coincidental, had potential for impacting the training
program. We found that many of the changes had potential for very
subtle impact; for example, the staff's optimism for the program
probably improved their teaching, but this notion is difficult to
substantiate. Also additional out- of-class study aids had been
developed and introduced throughout the training program.
Cumulatively, one would expect these changes to result in improved
student performance; however, we felt the attempt to isolate and
attribute effects to individual factors would be impossible'in a -.
post hoc evaluation design.
With these ideas in mind, we approached the evaluation with
two broad questions: (1) How did the performance of pre-Model
School project students compare with the performance of post- Model
School students, in terms of attrition rate, setback (repeating of
course segments) rate, test scores, and number of retests? (2)
What changes occurred in the intervening time period which may have
impacted student performance?
Next we constructed baseline data for a group of students
attending the training prior to modification for use as a quasicontrol
group. The school had been utilizing an automated testing
program which maintained students' scores on tests and number of
retests taken. This data gave us a picture of performance in the
individual content areas, as well as an overall measure of
performance. We also collected more general performance measures
such as course attrition and setback data.
Corresponding information was collected for a comparison group
who received the training after Model School Program
implementation. Because academic ability levels of students in
fundamental training courses have historically varied
systematically with the season of the year, we selected our
comparison group from months corresponding to that of the '8contro111
group in an attempt to maximize equivalency of the two groups.
Data were, analyzed and major findings presented in the summary
format shown in Figure 1. This graphic representation enabled us
to overlay potential impacting factors with major measures of
student performance. Our clients liked this format because it
provided an at-a-glance picture of both the changes to the training
program and corresponding variations in terms of student
performance. In this instance, the overall impact of the program
appeared to be positive in that the two major indicators of
training success, attrition and setback rates, both improved.
125