Russel-Research-Method-in-Anthropology

Research Design: Experiments and Experimental Thinking 113
But a lot of what’s really powerful about the experimental method is
embodied in this example. Suppose that the rock-music group does better on
the task. We can be pretty sure this outcome is not because of the participants’
sex, age, or education, but because of the music. Just sticking in more independent
variables (like expanding the group to include men, graduate students,
or high school students; or playing different tunes; or making the learning task
more realistic), without modifying the experiment’s design to control for all
those variables, creates what are called confounds. They confound the experiment
and make it impossible to tell if the intervention is what really caused
any observed differences in the dependent variable.
Good experiments test narrowly defined questions. This is what gives them
knowledge-making power. When you do a good experiment, you know something
at the end of it. In this case, you know that women students at one school
memorize or do not memorize three-digit numbers better when they listen to
a particular rock tune.
This may not seem like much, but you really know it. You can repeat the
experiment at the same school to verify or refute this little bit of knowledge.
You can repeat the experiment at another school to see if the knowledge has
external validity.
Suppose you don’t get the same answer at another school, holding all the
other elements of the experiment—age, sex, type of music—constant. The
new finding demands an explanation. Perhaps there is something about the
student selection process at the two schools that produces the different results?
Perhaps students at one school come primarily from working-class families,
while students from the other school come from upper-middle-class families.
Perhaps students from different socioeconomic classes grow up with different
study habits, or prefer different kinds of music.
Conduct the experiment again but include men this time. Conduct it again
and include two music conditions: a rock tune and a classical piece. Take the
experiment on the road and run it all over again at different-sized schools in
different regions of the country. Then, on to Paraguay. . . .
True experiments, with randomized assignment and full control by the
researcher, produce knowledge that has high internal validity. This means
that changes in the dependent variables were probably caused by—not merely
related to or correlated with—the treatment. Continual replication produces
cumulative knowledge with high external validity—that is, knowledge that
you can generalize to people who were not part of your experiment.
Replication of knowledge is every bit as important as its production in the
first place. In fact, it terms of usefulness, replicated knowledge is exactly what
we’re after.