NAVY SCHOOL MANAGEMENT MANUAL

Checklists. Checklists should be treated in the same way
as rating scales, except there will be only two possible
scores, 1 or 0, yes or no, or go or no-go. Again,
different raters should be compared with each other, as in
the table above, to determine if there is substantial
disagreement. If so, the checklists and instructions to
scorers should be reviewed.
Essay Test Items. Since essay items are best scored using
checklists or rating scales for major points in the
answers, the procedures described above are applicable.
Test Analysis for Selected-Response Items. Analyzing the
individual test item is only part of the analysis process.
Statistically, individual test items may appear acceptable, but
when placed together on the test, may not accomplish what the
test was designed to do. This section provides some guidelines
to consider when conducting test analysis.
Content Validity is defined as the extent to which a test
measures the objectives. Tests should have content
validity prior to conducting the test item analysis.
Measures of Central Tendency are statistical measures
commonly referred to as the mean, median and mode. The
median is the middle grade. The mode is the most frequent
grade in the sample. The mean is the average grade. It is
the statistic to be most concerned with during test
analysis.
The mean grade on a test provides information on the
average student. If the minimum passing score for a
test is set at 70 and the mean is 70, then the average
student is achieving the minimum score. If this is
occurring, then the minimum passing score is probably
set too high.
When the mean grade is low, it could indicate that the
test is too difficult, leading to student frustration.
On the other hand, if the mean grade is 95, the test
is probably not challenging enough for students.
When using the mean, be aware that the grade may not
accurately reflect student performance since it may be
affect by extremely high and low scores.
Validation requires a list of the number, type, and
knowledge levels for all test items in a course. If a test
was designed properly, there is a greater chance that the
test has content validity and is therefore measuring the
C-20
NAVEDTRA 135C