Contents

CHAPTER 10: Quasi-Experimental Designs and Program Evaluation 321In addition to problems resulting from threats to internal validity, trueexperiments can be weakened by threats to external validity. External validitydepends mainly on how representative our sample is of the persons, settings,and times to which we want to generalize. Representativeness is normallyachieved through random sampling. Because random sampling is used so infrequently(see Shadish et al., 2002), however, we can rarely say that our sample ofparticipants, or the situation in which we are making observations, or the timesduring which we test individuals are representative samples of all persons, settings,treatments, or outcomes. Therefore, the investigator must be aware ofpossible interactions between the independent variable of an experiment and,for example, the type of individual or the nature of the setting that is involvedin the experiment. Is a difference, for instance, between an experimental groupand a control group that is observed with volunteers from an inner-city schoolin the winter also likely to be found when nonvolunteers are tested in a suburbanschool in the spring of the year?Cook and Campbell describe several approaches to evaluating threats toexternal validity; the most important is attempting to determine the representativenessof the sample. They point out, however, that the best test of externalvalidity is replication. Thus, the question of external validity is best answeredby repeating the experiment with different types of participants, in differentsettings, with different treatments, and at different times. Occasionally partialreplications can be “built into” an experiment—for example, by selecting morethan one group to participate. Testing schoolchildren from a lower socioeconomicgroup and a higher socioeconomic group in an experiment designedto determine the effectiveness of a new educational program would provideevidence of the generality of the treatment’s effectiveness across these twosocio economic groups.QUASI-EXPERIMENTS• Quasi-experiments provide an important alternative when true experimentsare not possible.• Quasi-experiments lack the degree of control found in true experiments;most notably, quasi-experiments typically lack random assignment.• Researchers must seek additional evidence to eliminate threats to internalvalidity when they do quasi-experiments rather than true experiments.• The one-group pretest-posttest design is called a pre-experimental designor a bad experiment because it has so little internal validity.Key ConceptA dictionary will tell you that one definition of the prefix quasi- is “resembling.”Quasi-experiments involve procedures that resemble those of true experiments.Generally speaking, quasi-experiments include some type ofintervention or treatment and they provide a comparison, but they lack thedegree of control found in true experiments. Just as randomization is the hallmarkof true experiments, so lack of randomization is the hallmark of quasiexperiments.As Campbell and Stanley (1966) explain, quasi-experiments arisewhen researchers lack the control necessary to perform a true experiment.