RESEARCH METHOD COHEN ok

430 TESTS
the test and its component items. With regard to
the former, in part this is a matter of reliability,
for the time of day or week etc. might influence
how alert, motivated or capable a student might
be. With regard to the latter, the researcher will
need to decide what time restrictions are being
imposed and why; for example, is the pressure
of a time constraint desirable – to show what
a student can do under time pressure – or an
unnecessary impediment, putting a time boundary
around something that need not be bounded (was
Van Gogh put under a time pressure to produce
paintings of sunflowers) (see also Kohn 2000).
Although it is vital that students know what
the overall time allowance is for the test, clearly
it might be helpful to indicate notional time
allowances for different elements of the test; if
these are aligned to the relative weightings of
the test (see the discussions of weighting and
scoring) they enable students to decide where
to place emphasis in the test – they may want
to concentrate their time on the high scoring
elements of the test. Further, if the items of the
test have exact time allowances, this enables a
degree of standardization to be built into the test,
and this may be useful if the results are going to be
used to compare individuals or groups.
Plan the scoring of the test
The awarding of scores for different items of the
test is a clear indication of the relative significance
of each item – the weightings of each item are
addressed in their scoring. It is important to
ensure that easier parts of the test attract fewer
marks than more difficult parts of it, otherwise a
student’s results might be artificially inflated by
answering many easy questions and fewer more
difficult questions (Gronlund and Linn 1990).
Additionally, there are several attractions to
making the scoring of tests as detailed and specific
as possible (Cresswell and Houston 1991; Gipps
1994; Aiken 2003), awarding specific points for
each item and sub-item, for example:
It enables partial completion of the task to be
recognized – students gain marks in proportion
to how much of the task they have completed
successfully (an important feature of domainreferencing).
It enables a student to compensate for doing
badly in some parts of a test by doing well in
other parts of the test.
It enables weightings to be made explicit to
the students.
It enables the rewards for successful completion
of parts of a test to reflect considerations such
as the length of the item, the time required to
complete it, its level of difficulty, its level of
importance.
It facilitates moderation because it is clear and
specific.
It enables comparisons to be made across groups
by item.
It enables reliability indices to be calculated
(see discussions of reliability).
Scores can be aggregated and converted into
grades straightforwardly.
Ebel (1979) argues that the more marks
that are available to indicate different levels of
achievement (e.g. for the awarding of grades),
the greater the reliability of the grades will
be, although clearly this could make the test
longer. Scoring will also need to be prepared
to handle issues of poor spelling, grammar and
punctuation – is it to be penalized, and how will
consistency be assured here Further, how will
issues of omission be treated, e.g. if a student omits
the units of measurement (miles per hour, dollars
or pounds, meters or centimetres)
Related to the scoring of the test is the issue
of reporting the results. If the scoring of a test is
specific then this enables variety in reporting to
be addressed, for example, results may be reported
item by item, section by section, or whole test
by whole test. This degree of flexibility might be
useful for the researcher, as it will enable particular
strengths and weaknesses in groups of students to
be exposed.
The desirability of some of the above points
is open to question. For example, it could be
argued that the strength of criterion-referencing is
precisely its specificity, and that to aggregate data