2003 IMTA Proceedings - International Military Testing Association

Exam Center to work with the project full-time. The third lead member was a relative newcomer
to the Center and dedicated half-time to the project. Their backgrounds included engineering,
computer programming, instructional design, project management, procurement, and human
performance expertise. Project responsibilities fell into three general areas - hardware, software
and content – with one lead member responsible for each. Process issues tended to cross all
three of these general areas and were addressed both by individuals and the Team as a whole.
The lead Team tapped expertise from other departments as required for short-term guidance and
assistance.
Second, every effort was made to achieve a series of rapid successes in small, but critical,
elements of the electronic exam process to prove the concept, gain acceptance, build momentum
and establish ultimate success as a realistic possibility. As Phase II was initiated, a number of
individuals expressed concern and voiced their reasons why this initiative would fail. The Team
moved quickly to review and select hardware and software that, out of the box, addressed key
questions people had about the viability of creating and delivering electronic exams. With Team
coordination and assistance from the Exam Development Software (EDS) programmer, required
modifications that were anticipated by some to take three to six months were achieved to an 80
percent level in two weeks, with a prototype ready for implementation in less than one month.
SME content developers moved quickly to generate sample multimedia exam questions that were
performance-oriented and illustrated content that would have been difficult or impossible to
convey through text alone. Key employees and persons of influence within and associated with
the Navy Advancement Center were kept informed of progress through regular meetings held
every two weeks. The short time frame within which these successes were achieved started to
move the organizational culture from a “This will never happen” position to “This initiative is
going to happen and we need to be a part of it.”
Finally, efforts were undertaken to brief higher-level echelons on the plan and solicit
support. Tying this initiative to the Navy’s current Revolution in Training with its emphasis on
human performance was key. The same infrastructure that will allow Navy members electronic
access to elements of their career progression can be used to provide Navy advancement testing
in an all-electronic format. While implementation of this access is not expected for some time, it
is important to work out the details of the electronic advancement testing process now, so that it
can be expanded as soon as the infrastructure is in place.
PROCESS ISSUES
The United States Navy prides itself on its track record of consistently creating and
administering examinations to rank order enlisted Sailors for advancement in the grades E4-E7.
To smooth the transition, Phase II processes adhered to the existing paper-based processes to the
greatest extent practicable. Review of the processes currently used for the paper and pencil
examination process indicated that only minor modification was needed to accommodate the
electronic testing process. For example, electronic testing will utilize existing Education
Services Officers (ESOs), who will administer the exams in a manner consistent with the current
paper and pencil process. Where variations between the processes exist (like charging computer
batteries the night before the exam), these additional responsibilities will be clearly
communicated. Following the exam, in non-networked environments the ESO will express-mail
413
45 th Annual Conference of the International Military Testing Association
Pensacola, Florida, 3-6 November 2003