RESEARCH METHOD COHEN ok

DOCUMENTARY RESEARCH 203
What are you, the reader/researcher bringing
to the document in trying to make sense of it
What alternative interpretations of the
document are possible and tenable How is
the chosen interpretation justified
What are the problems of reliability and
validity in your reading of the document
What is the place of the document in the
overall research project
Questions are being raised here about the reliability
and validity of the documents (see http://
www.routledge.com/textbooks/9780415368780 –
Chapter 8, file 8.3. ppt). They are social products,
located in specific contexts, and, as such, have
to be interrogated and interpreted rather than
simply accepted. They are often selective, deliberately
excluding certain details or information
and serving purposes and audiences other than the
researcher. Documents lie on several continua, for
example:
Formal/official ↔ Informal/lay
Published ↔ Unpublished
Public domain ↔ Private papers
Anonymous ↔ Authored
Facts ↔ Beliefs
Professional ↔ Lay
For circulation ↔ Not for circulation
Placing documents along these several continua
can assist the researcher in answering the
preceding long list of questions.
Reliability and validity in documentary
analysis
Validity may be strong in first person documents
or in documents that were written for a specific
purpose (Bailey 1994: 317). However, that purpose
may not coincide with that of research, thereby
undermining its validity for research purposes. We
mentioned earlier the problem of bias, selectivity,
being written for an audience and purposes
different from those of the researcher, attrition and
selective survival; all these undermine validity.
In historical research great care is paid to
authenticity and provenance, and documents may
be subject to chemical analysis here (e.g. of
inks, paper, parchment and so on) in order to
detect forgeries. Bailey (1994: 318) suggests that
face validity and construct validity in documents
may be stronger and more sufficient than other
forms of validity, though corroboration with
other documents should be undertaken wherever
possible.
With regard to reliability, while subjectivity may
feature highly in certain documents, reliability
by corroboration may also be pursued. The
standards and criteria of reliability have to
be declared by the researcher. Scott (1990)
suggests four criteria for validity and reliability
in using documents: authenticity; credibility
(including accuracy, legitimacy and sincerity);
representativeness (including availability and
which documents have survived the passage of
time); and meaning (actual and interpreted).
It is often difficult to disentangle fact from
interpretation in a document and the research
that is conducted using it (see http://www.
routledge.com/textbooks/9780415368780 –
Chapter 8, file 8.4. ppt). Understanding documents
is a hermeneutic exercise, at several
stages. Giddens (1979) remarked that researchers
have to live with a ‘double hermeneutic’, that is,
they interpret a world that is already interpreted
by the participants, a pre-interpreted world. Actors
or participants interpret or ascribe meaning to
the world and then the researcher interprets or ascribes
meaning to these interpretations. However,
for the user of documents, the matter extends
further. Documents record live events, so written
data on social events become second hand
because they translate the researcher’s/writer’s
interpretation/inference of the world into another
medium – from action to writing: a triple
hermeneutic. Documents are part of the world
and the action on which they are commenting.
Then the reader places his or her interpretation/inference
on the document, a quadruple
hermeneutic. At each of these four stages interpretation,
inference and bias and, thereby,
unreliability could enter the scene. As Connelly
and Clandinin (1997: 84) remark, converting field
text into a research text is a process of (increasing)
interpretation. Field texts and documents, they
Chapter 8