RESEARCH METHOD COHEN ok

ENSURING VALIDITY 145
avoiding having too long or too short an
interval between pretests and post-tests
ensuring inter-rater reliability
matching control and experimental groups
fairly
ensuring standardized procedures for gathering
data or for administering tests
building on the motivations of the respondents
tailoring the instruments to the concentration
span of the respondents and addressing other
situational factors (e.g. health, environment,
noise, distraction, threat)
addressing factors concerning the researcher
(particularly in an interview situation);
for example, the attitude, gender, race,
age, personality, dress, comments, replies,
questioning technique, behaviour, style and
non-verbal communication of the researcher.
At the stage of data analysis there are several
areas where invalidity lurks; these might be
minimized by:
using respondent validation
avoiding subjective interpretation of data (e.g.
being too generous or too ungenerous in the
award of marks), i.e. lack of standardization
and moderation of results
reducing the halo effect, where the researcher’s
knowledge of the person or knowledge of other
data about the person or situation exerts an
influence on subsequent judgements
using appropriate statistical treatments for
the level of data (e.g. avoiding applying
techniques from interval scaling to ordinal data
or using incorrect statistics for the type, size,
complexity, sensitivity of data)
recognizing spurious correlations and extraneous
factors which may be affecting the data
(i.e. tunnel vision)
avoiding poor coding of qualitative data
avoiding making inferences and generalizations
beyond the capability of the data to
support such statements
avoiding the equating of correlations and
causes
avoiding selective use of data
avoiding unfair aggregation of data (particularly
of frequency tables)
avoiding unfair telescoping of data (degrading
the data)
avoiding Type I and/or Type II errors (see
http://www.routledge.com/textbooks/
9780415368780 – Chapter 6, file 6.2.doc).
A Type I error is committed where the
researcher rejects the null hypothesis when it
is in fact true (akin to convicting an innocent
person: Mitchell and Jolley 1988: 121); this
can be addressed by setting a more rigorous
level of significance (e.g. ρ