RESEARCH METHOD COHEN ok

146 VALIDITY AND RELIABILITY
presenting the data without misrepresenting its
message
making claims which are sustainable by the
data
avoiding inaccurate or wrong reporting of data
(i.e. technical errors or orthographic errors)
ensuring that the research questions are
answered; releasing research results neither too
soon nor too late.
Having identified where invalidity lurks, the
researcher can take steps to ensure that, as far
as possible, invalidity has been minimized in all
areas of the research.
Reliability in quantitative research
The meaning of reliability differs in quantitative
and qualitative research (see http://
www.routledge.com/textbooks/9780415368780 –
Chapter 6, file 6.4 ppt). We explore these concepts
separately in the next two sections. Reliability in
quantitative research is essentially a synonym for
dependability, consistency and replicability over
time, over instruments and over groups of respondents.
It is concerned with precision and accuracy;
some features, e.g. height, can be measured precisely,
while others, e.g. musical ability, cannot.
For research to be reliable it must demonstrate
that if it were to be carried out on a similar
group of respondents in a similar context
(however defined), then similar results would be
found. There are three principal types of reliability:
stability, equivalence and internal consistency
(see http://www.routledge.com/textbooks/
9780415368780 – Chapter 6, file 6.5. ppt).
Reliability as stability
In this form reliability is a measure of consistency
over time and over similar samples. A reliable
instrument for a piece of research will yield
similar data from similar respondents over time.
Aleakingtapwhicheachdayleaksonelitreis
leaking reliably whereas a tap which leaks one litre
some days and two litres on others is not. In the
experimental and survey models of research this
would mean that if a test and then a retest were
undertaken within an appropriate time span, then
similar results would be obtained. The researcher
has to decide what an appropriate length of time is;
too short a time and respondents may remember
what they said or did in the first test situation,
too long a time and there may be extraneous
effects operating to distort the data (for example,
maturation in students, outside influences on the
students). A researcher seeking to demonstrate
this type of reliability will have to choose an
appropriate time scale between the test and retest.
Correlation coefficients can be calculated for the
reliability of pretests and post-tests, using formulae
which are readily available in books on statistics
and test construction.
In addition to stability over time, reliability as
stability can also be stability over a similar sample.
For example, we would assume that if we were
to administer a test or a questionnaire simultaneously
to two groups of students who were very
closely matched on significant characteristics (e.g.
age, gender, ability etc. – whatever characteristics
are deemed to have a significant bearing, on the
responses), then similar results (on a test) or responses
(to a questionnaire) would be obtained.
The correlation coefficient on this form of the
test/retest method can be calculated either for the
whole test (e.g. by using the Pearson statistic or a
t-test) or for sections of the questionnaire (e.g. by
using the Spearman or Pearson statistic as appropriate
or a t-test). The statistical significance of
the correlation coefficient can be found and should
be 0.05 or higher if reliability is to be guaranteed.
This form of reliability over a sample is particularly
useful in piloting tests and questionnaires.
In using the test-retest method, care has to be
taken to ensure (Cooper and Schindler 2001: 216)
the following:
The time period between the test and retest is
not so long that situational factors may change.
The time period between the test and retest is
not so short that the participants will remember
the first test.
The participants may have become interested
in the field and may have followed it up