RESEARCH METHOD COHEN ok

TYPES OF QUESTIONNAIRE ITEMS 331
Further, if a genuinely open-ended question is
being asked, it is perhaps unlikely that responses
will bear such a degree of similarity to each
other so as to enable them to be aggregated too
tightly. Open-ended questions make it difficult
for the researcher to make comparisons between
respondents, as there may be little in common to
compare. Moreover, to complete an open-ended
questionnaire takes much longer than placing a
tick in a rating scale response box; not only
will time be a constraint here, but there is an
assumption that respondents will be sufficiently or
equally capable of articulating their thoughts and
committing them to paper.
In practical terms, Redline et al. (2002)report
that using open-ended questions can lead to
respondents overlooking instructions, as they are
occupied with the more demanding task of writing
in their own words than reading instructions.
Despite these cautions, the space provided for an
open-ended response is a window of opportunity
for the respondent to shed light on an issue or
course. Thus, an open-ended questionnaire has
much to recommend it.
Matrix questions
Matrix questions are not types of questions but
concern the layout of questions. Matrix questions
enable the same kind of response to be given to
several questions, for example ‘strongly disagree’
to ‘strongly agree’. The matrix layout helps to save
space, for example:
Please complete the following by placing a tick in
one space only, as follows:
1 = not at all; 2 = very little; 3 = a moderate
amount; 4 = quite a lot; 5 = averygreatdeal
How much do you use the following for assessment
purposes
1 2 3 4 5
(a) commercially published tests [ ] [ ] [ ] [ ] [ ]
(b) your own made-up tests [ ] [ ] [ ] [ ] [ ]
(c) students’ projects [ ] [ ] [ ] [ ] [ ]
(d) essays [ ] [ ] [ ] [ ] [ ]
(e) samples of students’ work [ ] [ ] [ ] [ ] [ ]
Here five questions have been asked in only five
lines, excluding, of course, the instructions and
explanations of the anchor statements. Such a
layout is economical of space.
A second example indicates how a matrix
design can save a considerable amount of space
in a questionnaire. Here the size of potential
problems in conducting a piece of research is
asked for, and data on how much these problems
were soluble are requested. For the first issue
(the size of the problem), 1 = no problem, 2 = a
small problem, 3 = amoderateproblem,4= a
large problem, 5 = averylargeproblem.Forthe
second issue (how much the problem was solved),
1 = not solved at all, 2 = solved only a very little,
3 = solved a moderate amount, 4 = solved a lot,
5 = completely solved (see Box 15.4).
Here thirty questions (15 × 2) have been able
to be covered in just a short amount of space.
Laying out the questionnaire like this enables
the respondent to fill in the questionnaire rapidly.
On the other hand, it risks creating a mind set in
the respondent (a ‘response set’: Baker 1994: 181)
in that the respondent may simply go down the
questionnaire columns and write the same number
each time (e.g. all number 3) or, in a rating
scale, tick all number 3. Such response sets can
be detected by looking at patterns of replies and
eliminating response sets from subsequent analysis.
The conventional way of minimizing response
sets has been by reversing the meaning of some
of the questions so that the respondents will need
to read them carefully. However, Weems et al.
(2003) argue that using positively and negatively
worded items within a scale is not measuring
the same underlying traits. They report that
some respondents will tend to disagree with
a negatively worded item, that the reliability
levels of negatively worded items are lower
than for positively worded items, and that
negatively worded items receive greater nonresponse
than positively worded items. Indeed
Weems et al. (2003) argue against mixed-item
formats, and supplement this by reporting that
inappropriately worded items can induce an
artificially extreme response which, in turn,
compromises the reliability of the data. Mixing
Chapter 15