Contents

CHAPTER 10: Quasi-Experimental Designs and Program Evaluation 341Many social scientists have argued that society must be willing to take anexperimental approach to social reform—one that will allow the clearest evaluationof the effectiveness of new programs. In many situations (for instance,when available resources are scarce), true experiments involving randomizationof individuals to treatment and no-treatment conditions are recommended.However, if a true experiment is not feasible, quasi-experimental proceduresare the next best approach. Quasi-experiments differ from true experimentsin that fewer plausible rival hypotheses for an experimental outcome are controlled.When specific threats to the internal validity of an experiment arenot controlled, then the experimenter, by logically examining the situation andby collecting additional evidence, must seek to rule out these threats to internalvalidity.A particularly strong quasi-experimental procedure is the nonequivalentcontrol group design. This procedure generally controls for all major threatsto internal validity except those associated with additive effects of (1) selectionand history, (2) selection and maturation, (3) selection and instrumentation,and (4) threats due to differential statistical regression. In addition to themajor threats to internal validity, an experimenter must be sensitive to possiblecontamination resulting from communication between groups of participants.Problems of experimenter expectancy effects (observer bias); questionsof external validity; and novelty effects, including the Hawthorne effect, arepotential problems in all experiments, whether conducted in the laboratory orin the field.When it is possible to observe changes in a dependent measure beforeand after a treatment is administered, one can carry out a simple interruptedtime-series design. The researcher using this design looks for an abruptchange (discontinuity) in the time series that coincides with the introductionof the treatment. The major threat to internal validity in this design ishistory—some event other than the treatment may have been responsiblefor the change in the time series. Instrumentation also can be a problem,especially when the treatment represents a type of social reform that maylead to changes in the way records are kept or data collected. By including acontrol group that is as similar as possible to the experimental group, one canstrengthen the internal validity of a simple time-series design. A time serieswith nonequivalent control group, for example, controls for many possiblehistory threats.A particularly important goal of research in natural settings is programevaluation. Professionals other than psychologists (such as educators, politicalscientists, and sociologists) are often involved in this process. Types of programevaluation include assessment of needs, process, outcome, and efficiency.Perhaps the most serious constraints on program evaluation are the politicaland social realities that surround it. The reluctance of public officials to seekan evaluation of social reforms is often an obstacle to be overcome. Nevertheless,social scientists have called on program evaluators to make themselvesavailable to human services organizations. By answering this call, we may helpchange society in a way that will bring the most effective services to those mostin need.