NAVY SCHOOL MANAGEMENT MANUAL

The frequency with which an analysis is conducted may vary.
While 100 answer sheets is the recommended number to use
for analysis, this may not always be possible or practical.
If a course has a large student input, conducting an
analysis for every 100 answer sheets may be too time
consuming. If this occurs, testing personnel may be
able to conduct a monthly analysis until the items are
considered stable. Once stable, the analysis can be
conducted on a quarterly basis.
If a course has a very small student input, and it may
take several years to collect 100 answer sheets. For
courses with small inputs, the entire sample may be
used to calculate the effectiveness of the
alternatives and the index of discriminating power.
• These courses may also use the 50 percent missed
rule. With this method, each test item that is
missed by 50 percent of the students is reviewed
for possible problem areas.
• If 100 answer sheets can be accumulated in a
year's time, then a complete analysis, using all
three indexes should be conducted. If not, then
a complete analysis may not be required.
• If a complete analysis is not required, the 50
percent missed rule applies.
The important thing is not so much that an analysis be
conducted every time 100 answer sheets are received,
but that an analysis is conducted and the results are
used to improve the instruction.
In the analysis of a test item, it is important to record
the date an item was changed or the date the instructional
materials were changed. Each time an item or material is
changed, the analysis must begin again. When this is done,
it is possible to compare the performance of the test item
before and after the change.
After the test items are analyzed, the next step is to make
decisions based on the data.
First, determine which items do not fall into the
acceptable indexes discussed earlier. Each item is
then reviewed by asking several questions:
• Is the answer miskeyed?
C-15
NAVEDTRA 135C