I__. - International Military Testing Association

INTRODUCTION
DEVELOPMENT OF EQUATIONS FOR PREDICTING
TESTING IMPORTANCE OF TASKS
Walter G. Albert
William J. Phalen
Air Force Human Resources Laboratory
The Specialty Knowledge Test (SKT) is an important component
of the Weighted Airman Promotion System (WAPS). SKTs are lOOitem
multiple choice achievement tests designed to measure job
knowledge in various Air Force Specialties (AFSs). They are
written annually for each AFS by teams of four to eight subject
matter experts (SMEs). The SMEs are senior NCOs in the AFS for
which a particular test is being written. A psychologist
experienced in test construction procedures is assigned to each
team to serve as a group facilitator.
A critical part of the test construction process for any SKT
is the preparation of the test outline, which guides the SMEs in
determining how many questions they should write for each
knowledge or duty area of the AFS. The outline used in test
construction is generated in one of two ways. For many years,
the SMEs created their own outline, which is referred to as the
Conventional Test Outline (CTO). Recently, an automated process
has been used to develop outlines for some AFSs. With this
process, the Automated Test Outline (ATO) is available for use
when the test development team arrives. The AT0 is generated
from information gathered from testing importance (TI) surveys,
where senior NCOs are asked to rate the importance of each task
as to whether the knowledge(s) required to perform it should be
covered by the SKT.
An important advantage of the AT0 procedure over the CT0
procedure is the direct link established between important tasks
performed by incumbents in the AFS and test questions which
address the knowledges required to perform those tasks. The AT0
process has been implemented in several AFSs, but currently it is
regarded as an experimental procedure and is being evaluated
against the CTO. This paper investigates whether information
routinely collected from occupational surveys can be used to
generate accurate TI values for each task. The resulting
prediction equations could then be used to select tasks for
inclusion in testing importance surveys of previously unsurveyed
AFSs or to serve as a surrogate for TI, when a TI survey cannot
be accomplished.
OCCUPATIONAL SURVEYS
An occupational inventory containing up to 2,000 task
statements is administered to a large number of incumbents in
each AFS. These tasks are grouped into seven to twenty duty
areas. Each duty area is comprised of a group of tasks that form
a major activity associated with the job specialty. Each
310
_ .