I__. - International Military Testing Association

a
----- --.___-__..-.. . . . _. ..____.._ -. ~~
.
cases, it will be necessary to introduce new data collection
instruments, e.g., surveys, questionnaires, etc.
Whether data collected should be restricted to only the area
modified, to a broader segment of the program, or to the program in
totality depends on the expected effect of the change. In general,
the interdependence of program parts usually warrants a complete
evaluation.
4. Monitor the implementation of the chancre. Keep informed about
how the implementation is proceeding. Document associated factors
which might impact the success of the change, such as changes in
instructor or student attitudes, changes in quality of either
instructors or students, or changes in resources.
5. After the nroaram has stabilized followina incorooration of the
chanae. aather data for comoarison with the baseline. This step
involves collecting data corresponding in type to the baseline data
for a sample of students under the modified program. This step
will involve readministration of instruments developed for the
evaluation, for example attitude questionnaires.
6. Analyze data and internret results. Most of our clients have
neither the time nor the propensity to wade through a morass of
statistics. Although we sometimes use fairly sophisticated
statistical procedures and usually include these analyses in the
report, we always attempt to synthesize the findings for our
clients. We try to answer the general question of how the training
program was affected by the modification in an easily accessible
one page (or one table or figure) summary.
MODEL APPLICATION
In 1987, a project known as the Model School program was
initiated at Electrician Mate's (EM) School at the Naval Training
Center, Great Lakes. The purpose of this project was to examine
the training program for EM's at this school, explore ways to make
that training better, and implement those that were feasible. As
a result of this project, a number of changes took place in this
program over the next two years. For instance, a technology-based
learning center was instituted, changes in remediation occurred,
the testing program was revised somewhat, etc.
In the spring of 1990, the Research Branch of the Navy
Technical Training Command was tasked with conducting an evaluation
of the impact of these changes on the training program. Because we
were not involved during the implementation of the project, we used
a modification of the six-step approach described above.
In this case, our first step was getting to know the program
we were tasked with evaluating. From talking with the school staff
124