A Guide to the ALM Thesis - iSites - Harvard University

20
| Chapter 2: The Research Process
Likewise, in education research it is common to compare one or more teaching approaches using naturally
occurring groups such as children in a specific school or students in two parallel sections of the same course.
(1) “ Scope of the observations” includes the breadth of the question: all the pieces of sculpture done by one
woman artist, a comparison of the works of several women artists from New York City over a fifty-year period,
or selected works that depict similar subject matter by only one artist? This is not just a question of how much
material will be covered; it includes also the direction of the research. Each of the three options presented has
a different slant. And for each slant, there is a necessary minimum (as well as a feasible maximum) amount
of material to be considered. In planning the thesis, students should be aware of pragmatic issues that may
interfere with obtaining sufficient data. Cost or time is not a sufficient reason to draw conclusions based on
inadequate samples or inappropriate methods.
In the biological, social, and behavioral sciences, this is directly related to the sample-size question. It boils
down to whether the researcher has an adequate and a representative amount of data for the number of
variables to be considered. The question to be answered and the type and amount of data collected must match.
Detailed, extensive observations of five subjects allow for very different conclusions than more specific and
precise measurements of one or two dimensions on 100 subjects. Five subjects are sufficient for a case series
using applied behavior analytic methods. One hundred subjects allow for an examination of group differences
or co-variations among several specific variables, e.g., an examination of how genotype predicts phenotypic
expressions of a particular characteristic. Trade-offs exist between intensive case studies of just a few individuals
and less intensive group studies of a larger population. Each method is appropriate to different kinds of
questions and the quality and depth of observations required for analysis. One of the things you will learn as you
familiarize yourself with the literature in any field is what constitutes a typical sample size.
(2) “ Investigator bias” refers to any effect on the research due to the experiences, prejudices, interests, and so
forth of the researcher or investigator. Since it is effectively impossible for us to be free of bias, the best we
can do is be aware of our biases and attempt to control or compensate for them. A researcher with a vested
interest in some particular outcome to the research may be markedly handicapped in interpreting the evidence.
Someone violently opposed to the Castro regime in Cuba, for example, may find it impossible to be sufficiently
objective to do thorough research on that country. Particularly but not only in experiments, the researcher
has expectations of results based upon the hypotheses and experimental manipulations. To control this,
experiments are typically run “blind,” that is, without the investigator knowing which subjects were assigned to
which experimental condition until all data have been coded.
On another and more common level, the researcher may be working with documents that are themselves
biased. US Civil War diaries will show very different reactions to and interpretations of given events, depending
on whether the author is from the North or South. Note, however, that this does not diminish the value of such
diaries. On the contrary, it is most useful to have more than one outlook or perspective on any matter or event.
But the researcher must be aware of these potential biases and be able to deal with research materials accordingly.
(3) “ Investigator effects” differ from investigator bias in that the latter refers to influences due to the investigator’s
mindset, while the former refers to influences due to unintended effects of procedures, to the observer’s physical
presence, and to manifestations of the observer. These effects are due to subject reactivity and are particularly
relevant when working with human subjects or animals. In essence, we all behave differently when we know we
are being observed, and in designing an investigation it is the researcher’s responsibility to mitigate this effect to
the extent possible. This may mean giving subjects time to acclimate to the experimental situation or observing
from some distance (e.g., using a one-way mirror or videotaping a subject from outside the room). In studies of
hypertension, “white coat hypertension” refers to an elevation in blood pressure that occurs in many people when
their blood pressure is assessed in a medical setting. To control for this effect, it is expected that a subject will be
asked to rest for 5 minutes prior to the assessment and then the measurement will be taken 3 times, allowing the
subject to get used to the experimenter, the equipment, and the sensations engendered by the procedure. Again,
A Guide to the ALM Thesis