RESEARCH METHOD COHEN ok

158 VALIDITY AND RELIABILITY
sensitivity of the questions asked, length of
the questionnaire)
issuing invitations to a follow-up interview
(face-to-face or by telephone)
providing encouragement to participate by a
friendly third party
understanding the nature of the sample
population in depth, so that effective targeting
strategies can be used.
The advantages of the questionnaire over interviews,
for instance, are: it tends to be more reliable;
because it is anonymous, it encourages greater honesty
(though, of course, dishonesty and falsification
might not be able to be discovered in a questionnaire);
it is more economical than the interview in
terms of time and money; and there is the possibility
that it can be mailed. Its disadvantages, on the
other hand, are: there is often too low a percentage
of returns; the interviewer is unable to answer questions
concerning both the purpose of the interview
and any misunderstandings experienced by the interviewee,
for it sometimes happens in the case of
the latter that the same questions have different
meanings for different people; if only closed items
are used, the questionnaire may lack coverage or
authenticity; if only open items are used, respondents
may be unwilling to write their answers
for one reason or another; questionnaires present
problems to people of limited literacy; and an interview
can be conducted at an appropriate speed
whereas questionnaires are often filled in hurriedly.
There is a need, therefore, to pilot questionnaires
and refine their contents, wording, length, etc. as
appropriate for the sample being targeted.
One central issue in considering the reliability
and validity of questionnaire surveys is that of
sampling. An unrepresentative, skewed sample,
one that is too small or too large can easily distort
the data, and indeed, in the case of very small
samples, prohibit statistical analysis (Morrison
1993). The issue of sampling was covered in
Chapter 4.
Validity and reliability in observations
There are questions about two types of validity in
observation-based research. In effect, comments
about the subjective and idiosyncratic nature of
the participant observation study are about its
external validity. How do we know that the results
of this one piece of research are applicable to
other situations Fears that observers’ judgements
will be affected by their close involvement in
the group relate to the internal validity of the
method. How do we know that the results of
this one piece of research represent the real
thing, the genuine product In Chapter 4 on
sampling, we refer to a number of techniques
(quota sampling, snowball sampling, purposive
sampling) that researchers employ as a way
of checking on the representativeness of the
events that they observe and of cross-checking
their interpretations of the meanings of those
events.
In addition to external validity, participant
observation also has to be rigorous in its internal
validity checks. There are several threats to
validity and reliability here, for example:
the researcher, in exploring the present, may
be unaware of important antecedent events
informants may be unrepresentative of the
sample in the study
the presence of the observer might bring about
different behaviours (reactivity and ecological
validity)
the researcher might ‘go native’, becoming too
attached to the group to see it sufficiently
dispassionately.
To address this Denzin (1970a) suggests
triangulation of data sources and methodologies.
Chapter 18 discusses the principal ways of
overcoming problems of reliability and validity
in observational research in naturalistic inquiry.
In essence it is suggested that the notion
of ‘trustworthiness’ (Lincoln and Guba 1985)
replaces more conventional views of reliability
and validity, and that this notion is devolved on
issues of credibility, confirmability, transferability and
dependability.Chapter18indicateshowtheseareas
can be addressed.
If observational research is much more
structured in its nature, yielding quantitative data,
then the conventions of intra- and inter-rater