RESEARCH METHOD COHEN ok

328 QUESTIONNAIRES
(Circle one number)
1 2 3 4 5 6
Strongly − − − − − − Strongly
agree
disagree
Let us say that one member of staff circled 1,
eight staff circled 2, twelve staff circled 3, nine
staff circled 4, two staff circled 5, and seven
staff circled 6. There being no mid-point on this
continuum, the researcher could infer that those
respondents who circled 1, 2 or 3 were in some
measure of agreement, while those respondents
who circled 4, 5 or 6 were in some measure of
disagreement. That would be very useful for, say,
a headteacher, in publicly displaying agreement,
there being twenty-one staff (1 + 8 + 12) agreeing
with the statement and eighteen (9 + 2 + 7)
displaying a measure of disagreement. However,
one could point out that the measure of ‘strongly
disagree’ attracted seven staff – a very strong
feeling – which was not true for the ‘strongly agree’
category, which attracted only one member of staff.
The extremity of the voting has been lost in a crude
aggregation.
Further, if the researcher were to aggregate the
scoring around the two mid-point categories (3
and 4) there would be twenty-one members of staff
represented, leaving nine (1 + 8) from categories
1and2andnine(2+ 7) from categories 5 and
6; adding together categories 1, 2, 5 and 6, a
total of eighteen is reached, which is less than the
twenty-one total of the two categories 3 and 4. It
seems on this scenario that it is far from clear that
there was agreement with the statement from the
staff; indeed taking the high incidence of ‘strongly
disagree’, it could be argued that those staff who
were perhaps ambivalent (categories 3 and 4),
coupled with those who registered a ‘strongly
disagree’, indicate not agreement but disagreement
with the statement.
The interpretation of data has to be handled
very carefully; ordering data to suit a researcher’s
own purposes might be very alluring but quite
illegitimate. The golden rule here is that crude data
can yield only crude interpretation; subtle statistics
require subtle data. The interpretation of data must
not distort the data unfairly. Rating scale questions
are treated as ordinal data (see Part Five), using
modal scores and non-parametric data analysis,
though one can find very many examples where
this rule has been violated, and non-parametric
data have been treated as parametric data. This is
unacceptable.
It has been suggested that the attraction of
rating scales is that they provide more opportunity
than dichotomous questions for rendering data
more sensitive and responsive to respondents. This
makes rating scales particularly useful for tapping
attitudes, perceptions and opinions. The need for a
pilot study to devise and refine categories, making
them exhaustive and discrete, has been suggested
as a necessary part of this type of data collection.
Questionnaires that are going to yield numerical
or word-based data can be analysed using computer
programmes (for example SPSS or Ethnograph
respectively). If the researcher intends to process
the data using a computer package it is essential
that the layout and coding system of the
questionnaire are appropriate for that particular
computer package. Instructions for layout in order
to facilitate data entry are contained in manuals
that accompany such packages.
Rating scales are more sensitive instruments
than dichotomous scales. Nevertheless, they are
limited in their usefulness to researchers by their
fixity of response caused by the need to select from
agivenchoice.Aquestionnairemightbetailored
even more to respondents by including open-ended
questions to which they can reply in their own
terms and own opinions. We consider these later.
Constant sum questions
In this type of question respondents are asked to
distribute a given number of marks (points) between
a range of items (see http://www.routledge.
com/textbooks/9780415368780 – Chapter 15, file
15.8. ppt). For example:
Please distribute a total of 10 points among the
sentences that you think most closely describe your
behaviour. You may distribute these freely: they may
be spread out, or awarded to only a few statements,
or all allocated to a single sentence if you wish.