Contents

CHAPTER 10: Quasi-Experimental Designs and Program Evaluation 323STRETCHING EXERCISEIn this exercise we ask you to consider possiblethreats to internal validity in this brief descriptionof a one-group pretest-posttest design.A psychologist interested in the effect of anew therapy for depression recruited a sampleof 20 individuals who sought relief from their depression.At the beginning of the study he askedall participants to complete a questionnaireabout their symptoms of depression. The meandepression score for the sample was 42.0 (thehighest possible score is 63.0), indicating severedepressive symptoms. (Individuals who are notdepressed typically score in the 0 to 10 rangeon this measure.) During the next 16 weeks thepsycho logist treated participants in the studywith the new treatment. At the end of the treatmentthe participants completed the depressionquestionnaire again. The mean score for theposttest was 12.0, indicating that, on average,participants’ depression symptoms were dramaticallyreduced and indicated only mild depression.The psychologist concluded that thetreatment was effective; that is, the treatmentcaused their depressive symptoms to improve.Cause-and-effect statements, such as theone made by this psychologist, are essentiallyimpossible to make when the one-group pretestposttestdesign is used. To understand why thisis true, we ask you to think of potential threats tointernal validity in this study.1 How might a history effect threaten the internalvalidity of this study?2 Explain how maturation likely plays a role in thisstudy.3 Are testing and instrumentation threats likely in thisstudy?4 Explain how statistical regression might influencethe interpretation of these findings.• If the two groups are similar in their pretest scores prior to treatment butdiffer in their posttest scores following treatment, researchers can moreconfidently make a claim about the effect of treatment.• Threats to internal validity due to history, maturation, testing,instrumentation, and regression can be controlled in a nonequivalentcontrol group design.Key ConceptThe one-group pretest-posttest design can be modified to create a quasiexperimentaldesign with greatly superior internal validity if two conditionsare met: (1) there exists a group “like” the treatment group that can serve as acomparison group, and (2) there is an opportunity to obtain pretest and posttestmeasures from individuals in both the treatment and the comparison groups.Campbell and Stanley (1966) call a quasi-experimental procedure that meetsthese two conditions a nonequivalent control group design. Because a comparisongroup is selected on bases other than random assignment, we cannot assumethat individuals in the treatment and control groups are equivalent on allimportant characteristics (i.e., a selection threat arises). Therefore, it is essentialthat a pretest be given to both groups to assess their similarity on the dependentmeasure. A nonequivalent control group design can be outlined as follows:O 1 X O 2- - - - - -O 1 O 2