2003 IMTA Proceedings - International Military Testing Association

exam. Gateway was able to provide these systems, configured to Phase II specifications, at close
to the target price.
Hardware Configuration Considerations
In evaluating and selecting a suitable hardware platform, decisions had to be made
regarding the specific configuration that would best support the delivery and administration of
the examinations. At the same time, the processes developed must place the fewest possible
demands on the exam proctors, as they likely will have little familiarity with the exam software
and may not have extensive computer experience. After careful consideration of a host of
options, the Phase II exam deployment strategy selected provides the exam on separate media, to
be plugged into the exam station at the time of the test. While various digital media were
analyzed, Secure Digital (SD) cards were ultimately selected for their economy, wide availability
and broad industry support. The Gateway M305E has an internal digital media reader accepting
SD cards. Using the SD card addressed three primary processes. First, it provided a way to get
the exam to the Sailor initially, and also established a fallback process that can be used if a Sailor
misses an exam and needs a makeup or alternate test. A second SD card with a new test can be
sent, rather than a new notebook computer. Second, it provided a way to get the exam back from
the Sailor. The concept of having exam proctors download answer files and e-mail them back to
the Center is attractive, but represents a significant change from current practice and presents
special challenges for exams with classified content. When SD cards are used to deliver the
exam, the results can be written back to the card, providing the administering activities with a
removable component that can be express-mailed back to the Navy Advancement Center in a
manner similar to that used for paper answer sheets. Finally, providing the test on SD cards
reduces the classification security storage concerns by allowing the computers to be shipped as
unclassified devices. Only the SD cards have to be stored in a classified safe prior to the day of
the exam. The computer systems themselves would only become classified once the exam was
inserted into the machine at test time, reducing the time the computers were classified devices
from several weeks to a few days. Classified storage issues could possibly be avoided
completely if the administering activities repackaged and returned the computer systems to the
Center immediately upon conclusion of the exam. If that is not possible, one solution is to
remove the computer hard drives and store them separately. This returns the computers to an
unclassified status free from security controls and reduces physical storage requirements.
SOFTWARE ISSUES
In Phase I, Perception for Windows by Questionmark was selected as the authoring
tool and exam delivery vehicle, “primarily because of its widespread commercial use and its
success in Navy training for end-of-course testing” (Baisden and Rymsza, 2002). As in the case
of the hardware, Phase II began with a review of available assessment software to determine
whether significant advances had occurred since Phase I. Two software packages were
determined to be the leading candidates: OutStart Evolution ® (a Learning Content Management
System (LCMS) that has been adopted by the Naval Education and Training Command (NETC)
as the primary content development tool for use with Navy E-Learning) and, once again,
415
45 th Annual Conference of the International Military Testing Association
Pensacola, Florida, 3-6 November 2003