RESEARCH METHOD COHEN ok

276 EXPERIMENTS AND META-ANALYSIS
and, hence, inferring causality is more contestable,
but they have the attraction of taking place in a
natural setting. Extraneous variables may include,
for example:
participant factors: they may differ on important
characteristics between the control and
experimental groups
intervention factors: the intervention may not
be exactly the same for all participants, varying,
for example, in sequence, duration, degree
of intervention and assistance, and other
practices and contents
situational factors: the experimental conditions
may differ.
These can lead to experimental error, in which
the results may not be due to the independent
variables in question.
The pretest-post-test control and
experimental group design
A complete exposition of experimental designs
is beyond the scope of this chapter. In
the brief outline that follows, we have selected
one design from the comprehensive treatment
of the subject by Campbell and Stanley
(1963) in order to identify the essential features
of what they term a ‘true experimental’
and what Kerlinger (1970) refers to as a ‘good’
design. Along with its variants, the chosen design
is commonly used in educational experimentation
(see http://www.routledge.com/textbooks/
9780415368780 – Chapter 13, file 13.3. ppt).
The pretest-post-test control group design can
be represented as:
Experimental RO 1 X O 2
Control RO 3 O 4
Kerlinger (1970) observes that, in theory, random
assignment to E and C conditions controls all
possible independent variables. In practice, of
course, it is only when enough subjects are
included in the experiment that the principle
of randomization has a chance to operate as
a powerful control. However, the effects of
Box 13.2
The effects of randomization
Select twenty cards from a pack, ten red and ten black.
Shuffle and deal into two ten-card piles. Now count the
number of red cards and black cards in either pile and
record the results. Repeat the whole sequence many
times, recording the results each time.
You will soon convince yourself that the most likely
distribution of reds and blacks in a pile is five in each:
the next most likely, six red (or black) and four black
(or red); and so on. You will be lucky (or unlucky for the
purposes of the demonstration!) to achieve one pile of
red and the other entirely of black cards. The probability
of this happening is 1 in 92,378. On the other hand, the
probability of obtaining a ‘mix’ of not more than six of
one colour and four of the other is about 82 in 100.
If you now imagine the red cards to stand for the
‘better’ ten children and the black cards for the ‘poorer’
ten children in a class of twenty, you will conclude that
the operation of the laws of chance alone will almost
probably give you close equivalent ‘mixes’ of ‘better’
and ‘poorer’ children in the experimental and control
groups.
Source:adaptedfromPilliner1973
randomization even with a small number of
subjects is well illustrated in Box 13.2.
Randomization, then, ensures the greater
likelihood of equivalence, that is, the apportioning
out between the experimental and control groups
of any other factors or characteristics of the
subjects which might conceivably affect the
experimental variables in which the researcher is
interested. If the groups are made equivalent, then
any so-called ‘clouding’ effects should be present
in both groups.
So strong is this simple and elegant true
experimental design, that all the threats to internal
validity identified in Chapter 6 are, according
to Campbell and Stanley (1963), controlled in
the pretest-post-test control group design. The
causal effect of an intervention can be calculated
in three steps:
1 Subtract the pretest score from the post-test
score for the experimental group to yield
score 1.