Technical Report - International Military Testing Association

Approaches to Improved Measurement:
Research in Progress
RCMLD K, GOODNIGHT (
US Army Enlfated Evaluation Center
Throughout the confer&e we have heard and discussed numerous topic6
epanning the realm of test immrovement and development. In every cane
each method used, the reaulto obtained, and the utilization of these
re6ults are all bared on reresrch. Reeearch is the guideline to BUECC~U
in almoet any endeavor. Tnncrefore, I am plcaeed to present to you several
of the more important reeearch projects which have been completed, as wel.1
68 nome currently in progrerit, at the US Army Enlirted Evaluation Center.
The first project, completed oeveral years ago, ir A Comparison of
Seven Methods of Computing Total Test Reliability fromma Single Terr
Adminietration. Ttdt reliability (that is--a determination of teat conriatency)
is evaluated by Lnvtstigating tht relationship of the praiictor
to itrtlf, or in other words, it is-the relationship of the ranking of
scores on one administration with the ranking on a aubsequant adminlstratfon.
The rsliabilfty of a test can generally be estimated adequately
fran only one administration of a test.
For tach MOS Evaluation Teat a reliability coefficient is computed
via the Kuder-ttfchardoori Formula 20. However, it ~68 imperative tc know
if thie wa6 the most accuratt indication of the consirtancy with which
the tert was measuring job proficiency. Thtrefore, this project was
designed to determine which of the odd-even, K-R 20 on the total group,
K-R 20 on 54% of the group, Hoyt’s Ana1ysi.e of VerCanct. Horat Maximum.
Horet Corrected, and Cleman’r Maximum reliabili*.v mearurement methods wan
super for. The result8 showed the K-R 20 relfability sethod on the total
group io the meet appropriate for the MOS Evaluation Te6ta, thus supporting
It8 uragt by thL Enlisted Evaluation Center (EEC).
Thi8 study was completed in February 1963, and prcaently another project,
A Canpariron of Fivt Hethods of Computing Total Ttrt Reliability frus
a Single Test Administration, ir under way. This 18 a replication of the
earlier study with some minor variation6 tc further verify EECI currant
procedure8 in tt6t reliability estimation. Also, another reason for con=
ducting this study lies in the proposed technical rtcommendationr which weft
presented at the 1964 American Psychological Association meeting&; their
rscamntndationr indicate that the K-R 21 ia more appropriate to use then the
K-R 20 under situations such as those at EEC. Therefore, in this rtudy five
methods of reliability eatimation-- K-R 20, K-R 21, odd-even, Hoyt’8 Analysis
of Variance, and Uorst’o Maximum--are beir.g compared to determine which is
the better method of mearuring te6t cons,etency in view of APA’s rtccmmendations
and to rtudy the effect8 of aample sire and MOS rkill level on the
varioue reliability coefficlentr. Results are not available thus far.
116
-_ ._.....- .--- .---. --.
c
_I__...