RESEARCH METHOD COHEN ok

CONSTRUCTING A TEST 423
where
A = the number of correct scores from the high
scoring group
B = the number of correct scores from the low
scoring group
N = the total number of students in the two
groups.
Suppose all 10 students from the high scoring
group answered the item correctly and 2 students
from the low scoring group answered the item
correctly. The formula would work out thus:
1
2
8
(10 + 10)
= 0.80 (index of discriminability)
The maximum index of discriminability is 1.00.
Any item whose index of discriminability is less
than 0.67, i.e. is too undiscriminating, should be
reviewed first to find out whether this is due to
ambiguity in the wording or possible clues in the
wording. If this is not the case, then whether the
researcher uses an item with an index lower than
0.67 is a matter of judgement. It would appear,
then, that the item in the example would be
appropriate to use in a test. For a further
discussion of item discriminability see Linn (1993)
and Aiken (2003).
One can use the discriminability index to
examine the effectiveness of distractors. Thisis
based on the premise that an effective distractor
should attract more students from a low scoring
group than from a high scoring group. Consider
the following example, where low and high scoring
groups are identified:
A B C
Top 10 students 10 0 2
Bottom 10 students 8 0 10
In example A, the item discriminates positively
in that it attracts more correct responses (10)
from the top 10 students than the bottom
10 (8) and hence is a poor distractor; here,
also, the discriminability index is 0.20, hence
is a poor discriminator and is also a poor
distractor. Example B is an ineffective distractor
because nobody was included from either group.
Example C is an effective distractor because
it includes far more students from the bottom
10 students (10) than the higher group (2).
However, in this case any ambiguities must be
ruled out before the discriminating power can be
improved.
Distractors are the stuff of multiple choice
items, where incorrect alternatives are offered, and
students have to select the correct alternatives.
Here a simple frequency count of the number
of times a particular alternative is selected will
provide information on the effectiveness of the
distractor: if it is selected many times then it is
working effectively; if it is seldom or never selected
then it is not working effectively and it should be
replaced.
If we wish to calculate the item difficulty of a test,
we can use the following formula:
A
N × 100
where
A = the number of students who answered the
item correctly;
N = the total number of students who attempted
the item.
Hence if 12 students out of a class of 20 answered
the item correctly, then the formula would work
out thus:
12
× 100 = 60 per cent
20
The maximum index of difficulty is 100 per cent.
Items falling below 33 per cent and above 67
per cent are likely to be too difficult and too
easy respectively. It would appear, then, that
this item would be appropriate to use in a test.
Here, again, whether the researcher uses an item
with an index of difficulty below or above the
cut-off points is a matter of judgement. In a normreferenced
test the item difficulty should be around
50 per cent (Frisbie 1981). For further discussion
of item difficulty see Linn (1993) and Hanna
(1993).
Given that the researcher can know the degree
of item discriminability and difficulty only once the
test has been undertaken, there is an unavoidable
Chapter 19