RESEARCH METHOD COHEN ok

DEFINING VALIDITY 137
generalization is problematical. For positivist
researchers generalizability is a sine qua non,while
this is attenuated in naturalistic research. For
one school of thought, generalizability through
stripping out contextual variables is fundamental,
while, for another, generalizations that say
little about the context have little that is
useful to say about human behaviour (Schofield
1990). For positivists variables have to be
isolated and controlled, and samples randomized,
while for ethnographers human behaviour is
infinitely complex, irreducible, socially situated
and unique.
Generalizability in naturalistic research is
interpreted as comparability and transferability
(Lincoln and Guba 1985; Eisenhart and Howe
1992: 647). These writers suggest that it is
possible to assess the typicality of a situation – the
participants and settings, to identify possible
comparison groups, and to indicate how data might
translate into different settings and cultures (see
also LeCompte and Preissle 1993: 348). Schofield
(1990: 200) suggests that it is important in
qualitative research to provide a clear, detailed
and in-depth description so that others can decide
the extent to which findings from one piece of
research are generalizable to another situation,
i.e. to address the twin issues of comparability
and translatability. Indeed, qualitative research
can be generalizable (Schofield 1990: 209), by
studying the typical (for its applicability to other
situations – the issue of transferability: LeCompte
and Preissle 1993: 324) and by performing
multi-site studies (e.g. Miles and Huberman
1984), though it could be argued that this is
injecting a degree of positivism into non-positivist
research. Lincoln and Guba (1985: 316) caution
the naturalistic researcher against this; they argue
that it is not the researcher’s task to provide
an index of transferability; rather, they suggest,
researchers should provide sufficiently rich data
for the readers and users of research to determine
whether transferability is possible. In this respect
transferability requires thick description.
Bogdan and Biklen (1992: 45) argue that
generalizability, construed differently from its
usage in positivist methodologies, can be addressed
in qualitative research. Positivist researchers, they
argue, are more concerned to derive universal
statements of general social processes rather than
to provide accounts of the degree of commonality
between various social settings (e.g. schools and
classrooms). Bogdan and Biklen (1992) are more
interested not with the issue of whether their
findings are generalizable in the widest sense but
with the question of the settings, people and
situations to which they might be generalizable.
In naturalistic research threats to external
validity include (Lincoln and Guba 1985:
189, 300):
selection effects: where constructs selected in
fact are only relevant to a certain group
setting effects: where the results are largely a
function of their context
history effects: where the situations have
been arrived at by unique circumstances and,
therefore, are not comparable
construct effects: where the constructs being
used are peculiar to a certain group.
Content validity
To demonstrate this form of validity the
instrument must show that it fairly and
comprehensively covers the domain or items
that it purports to cover. It is unlikely that
each issue will be able to be addressed in its
entirety simply because of the time available or
respondents’ motivation to complete, for example,
a long questionnaire. If this is the case, then
the researcher must ensure that the elements of
the main issue to be covered in the research
are both a fair representation of the wider issue
under investigation (and its weighting) and that
the elements chosen for the research sample
are themselves addressed in depth and breadth.
Careful sampling of items is required to ensure their
representativeness. For example, if the researcher
wished to see how well a group of students could
spell 1,000 words in French but decided to have a
sample of only 50 words for the spelling test, then
that test would have to ensure that it represented
the range of spellings in the 1,000 words – maybe
by ensuring that the spelling rules had all been
Chapter 6