RESEARCH METHOD COHEN ok

490 CONTENT ANALYSIS AND GROUNDED THEORY
Reliability in content analysis
There are several issues to be addressed in
considering the reliability of texts and their
content analysis, indeed, in analysing qualitative
data using a variety of means, for example:
Witting and unwitting evidence (Robson
1993: 273): witting evidence is that which was
intended to be imparted; unwitting evidence is
that which can be inferred from the text, and
which may not be intended by the imparter.
The text may not have been written with the
researcher in mind and may have been written
for a very different purpose from that of the
research (a common matter in documentary
research); hence the researcher will need to
know or be able to infer the intentions of the
text.
The documents may be limited, selective,
partial, biased, non-neutral and incomplete
because they were intended for a different
purpose other than that of research (an issue
of validity as well as of reliability).
It may be difficult to infer the direction of
causality in the documents – they may have
been the cause or the consequence of a
particular situation.
Classification of text may be inconsistent (a
problem sometimes mitigated by computer
analysis), because of human error, coder
variability (within and between coders), and
ambiguity in the coding rules (Weber 1990:
17).
Texts may not be corroborated or able to be
corroborated.
Words are inherently ambiguous and polyvalent
(the problem of homographs): for example,
what does the word ‘school’ mean: a building;
a group of people; a particular movement of
artists (e.g. the impressionist school); a department
(a medical school); a noun; a verb (to
drill, to induct, to educate, to train, to control,
to attend an institution); a period of instructional
time (‘they stayed after school to play
sports’); a modifier (e.g. a school day); a sphere
of activity (e.g. ‘the school of hard knocks’); a
collection of people adhering to a particular set
of principles (e.g. the utilitarian school); a style
of life (e.g. ‘a gentleman from the old school’);
a group assembled for a particular purpose (e.g.
agamblingschool),andsoon.Thisisaparticular
problem for computer programs which
may analyse words devoid of their meaning.
Coding and categorizing may lose the
nuanced richness of specific words and their
connotations.
Category definitions and themes may be
ambiguous, as they are inferential.
Some words may be included in the same
overall category but they may have more or
less significance in that category (and a system
of weighting the words may be unreliable).
Words that are grouped together into a similar
category may have different connotations and
their usage may be more nuanced than the
categories recognize.
Categories may reflect the researcher’s agenda
and imposition of meaning more than the text
may sustain or the producers of the text (e.g.
interviewees) may have intended.
Aggregation may compromise reliability.
Whereas sentences, phrases and words and
whole documents may have the highest
reliability in analysis, paragraphs and larger
but incomplete portions of text have lower
reliability (Weber 1990: 39).
A document may deliberately exclude something
for mention, overstate an issue or understate
an issue (Weber 1990: 73).
At a wider level, the limits of content analysis
are suggested by Ezzy (2002: 84), who argues
that, due to the pre-ordinate nature of coding
and categorizing, content analysis is useful for
testing or confirming a pre-existing theory rather
than for building a new one, though this perhaps
understates the ways in which content analysis can
be used to generate new theory, not least through
a grounded theory approach (discussed later). In
many cases content analysts know in advance what
they are looking for in text, and perhaps what the
categories for analysis will be. Ezzy (2002: 85)
suggests that this restricts the extent to which
the analytical categories can be responsive to the