RESEARCH METHOD COHEN ok

COMMERCIALLY PRODUCED TESTS AND RESEARCHER-PRODUCED TESTS 417
they may not be exactly suited to the purpose
required. Further, several commercially produced
tests have restricted release or availability, hence
the researcher might have to register with a
particular association or be given clearance to
use the test or to have copies of it. For example,
Harcourt Assessment and McGraw-Hill publishers
not only hold the rights to a world-wide battery
of tests of all kinds but also require registration
before releasing tests. In this example Harcourt
Assessment also has different levels of clearance,
so that certain parties or researchers may not be
eligible to have a test released to them because
they do not fulfil particular criteria for eligibility.
Published tests by definition are not tailored
to institutional or local contexts or needs; indeed
their claim to objectivity is made on the grounds
that they are deliberately supra-institutional. The
researcher wishing to use published tests must be
certain that the purposes, objectives and content of
the published tests match the purposes, objectives
and content of the evaluation. For example,
a published diagnostic test might not fit the
needs of the evaluation to have an achievement
test; a test of achievement might not have the
predictive quality that the researcher seeks in an
aptitude test, a published reading test might not
address the areas of reading that the researcher
is wishing to cover, a verbal reading test written
in English might contain language that is difficult
for a student whose first language is not English.
These are important considerations. A much-cited
text on evaluating the utility for researchers of
commercially available tests is produced by the
American Psychological Association (1999) in the
Standards for Educational and Psychological Testing
(http://www.apa.org/science/standards.html).
The golden rule for deciding to use a published
test is that it must demonstrate fitness for purpose.If
it fails to demonstrate this, then tests will have to
be devised by the researcher. The attraction of this
latter point is that such a ‘home-grown’ test will
be tailored to the local and institutional context
very tightly, i.e. that the purposes, objectives
and content of the test will be deliberately
fitted to the specific needs of the researcher in
aspecific,givencontext.Indiscussingfitnessfor
purpose, Cronbach (1949) and Gronlund and Linn
(1990) set out a range of criteria against which a
commercially produced test can be evaluated for
its suitability for specific research purposes.
Against these advantages of course there are several
important considerations in devising a ‘homegrown’
test. Not only might it be time-consuming
to devise, pilot, refine and then administer the
test but also, because much of it will probably be
non-parametric, there will be a more limited range
of statistics that may be applied to the data than
in the case of parametric tests.
The scope of tests and testing is far-reaching;
no areas of educational activity are untouched
by them. Achievement tests, largelysummativein
nature, measure achieved performance in a given
content area. Aptitude tests are intended to predict
capability, achievement potential, learning potential
and future achievements. However, the assumption
that these two constructs – achievement
and aptitude – are separate has to be questioned
(Cunningham 1998); indeed, it is often the
case that a test of aptitude for, say, geography at a
particular age or stage will be measured by using an
achievement test at that age or stage. Cunningham
(1998) has suggested that an achievement test
might include more straightforward measures of
basic skills, whereas aptitude tests might put these
in combination, for example combining reasoning
(often abstract) and particular knowledge; thus
achievement and aptitude tests differ according to
what they are testing.
Not only do the tests differ according to what
they measure, but also, since both can be used
predictively, they differ according to what they
might be able to predict. For example, because an
achievement test is more specific and often tied to a
specific content area, it will be useful as a predictor
of future performance in that content area but will
be largely unable to predict future performance
out of that content area. An aptitude test tends
to test more generalized abilities (e.g. aspects of
‘intelligence’, skills and abilities that are common
to several areas of knowledge or curricula), hence
it is able to be used as a more generalized predictor
of achievement. Achievement tests, Gronlund
(1985) suggests, are more linked to school
Chapter 19