RESEARCH METHOD COHEN ok

422 TESTS
importance of the knowledge or skill being tested,
the match of the item to the programme, and the
number of items to be included.
The basis of item analysis can be seen in
item response theory (see Hambleton 1993). Item
response theory (IRT) is based on the principle
that it is possible to measure single, specific latent
traits, abilities, attributes that, themselves, are not
observable, i.e. to determine observable quantities
of unobservable quantities. The theory assumes a
relationship between a person’s possession or level
of a particular attribute, trait or ability and his or
her response to a test item. IRT is also based on
the view that it is possible:
to identify objective levels of difficulty of
an item, e.g. the Rasch model (Wainer and
Mislevy 1990)
to devise items that will be able to discriminate
effectively between individuals
to describe an item independently of any
particular sample of people who might
be responding to it, i.e. is not group
dependent (i.e. the item difficulty and item
discriminability are independent of the sample)
to describe a testee’s proficiency in terms of
his or her achievement of an item of a known
difficulty level
to describe a person independently of any
sample of items that has been administered
to that person (i.e. a testee’s ability does not
depend on the particular sample of test items)
to specify and predict the properties of a test
before it has been administered;
for traits to be unidimensional (single traits are
specifiable, e.g. verbal ability, mathematical
proficiency) and to account for test outcomes
and performance
for a set of items to measure a common trait or
ability
for a testee’s response to any one test item not
to affect his or her response to another test
item
that the probability of the correct response
to an item does not depend on the number
of testees who might be at the same level of
ability
that it is possible to identify objective levels of
difficulty of an item
that a statistic can be calculated that indicates
the precision of the measured ability for each
testee, and that this statistic depends on the
ability of the testee and the number and
properties of the test items.
In constructing a test the researcher will need
to undertake an item analysis to clarify the item
discriminability and item difficulty of each item of
the test. Item discriminability refers to the potential
of the item in question to be answered correctly
by those students who have a lot of the particular
quality that the item is designed to measure and
to be answered incorrectly by those students who
have less of the particular quality that the same
item is designed to measure. In other words,
how effective is the test item in showing up
differences between a group of students Does the
item enable us to discriminate between students’
abilities in a given field An item with high
discriminability will enable the researcher to
see a potentially wide variety of scores on that
item; an item with low discriminability will show
scores on that item poorly differentiated. Clearly
a high measure of discriminability is desirable,
and items with low discriminability should be
discarded.
Suppose the researcher wishes to construct a test
of mathematics for eventual use with 30 students in
aparticularschool(orwithclassAinaparticular
school). The researcher devises a test and pilots
it in a different school or in class B respectively,
administering the test to 30 students of the same
age (i.e. the researcher matches the sample of the
pilot school or class to the sample in the school
which eventually will be used). The scores of the
30 pilot children are then split into three groups of
10 students each (high, medium and low scores).
It would be reasonable to assume that there will be
more correct answers to a particular item among
the high scorers than among the low scorers. For
each item compute the following:
A − B
1
2 (N)