Contents

206 PART III: Experimental MethodsBOX 6.2AN EXAMPLE OF META-ANALYSIS:“EVIDENCE-BASED YOUTH PSYCHOTHERAPIES VERSUSUSUAL CLINICAL CARE”Weisz, Jensen-Doss, and Hawley (2006) usedmeta-analysis to summarize the results of 32 psychotherapystudies with youth that comparedthe effects of “evidence-based treatments” and“usual care.” An evidence-based treatment (EBT)is one that has received empirical support—thatis, it has been shown in clinical research to helpindividuals. Although it seems obvious that EBTsshould be widely used in clinical practice becauseof this empirical support, many therapists arguethat these treatments would not be effective inusual clinical contexts. EBTs are structured andrequire therapists to follow a treatment manual.Some clinicians argue that EBTs are inflexible,rigid treatments that cannot be individualizedaccording to clients’ needs. Furthermore, opponentsof EBTs argue that empirical studies thatindicate effectiveness typically involve clients withless severe or less complicated problems thanthose seen in usual clinical practice. These argumentssuggest that usual care (UC) in the form ofpsychotherapy, counseling, or case managementas regularly conducted by mental health providerswould better meet the needs of the clientstypically seen in community settings.Weisz and his colleagues used meta-analysisto compare directly the outcomes associatedwith EBTs and usual care. Across 32 studies thatcompared EBT and UC, the average effect sizewas 0.30. Thus, youth treated with an evidencebasedtreatment were better off, on average, thanyouth treated with usual care. The value of 0.30falls between Cohen’s (1988) criteria for small andmedium effects. This effect size represents thedifference between the two types of treatments,not the effect of psychotherapy per se. Weiszet al. note that when EBTs are contrasted withno-treatment control groups (e.g., waiting list),the effect sizes for EBT typically range from 0.50to 0.80 (medium-to-large effects). In additionalanalyses the authors grouped studies accordingto factors such as the severity and complexity oftreated problems, treatment settings, and characteristicsof the therapists. These analyses weredone to determine whether the concerns voicedby critics of evidence-based treatments warrantthe continued use of usual care. Weisz and hiscolleagues found that grouping studies accordingto these various factors did not influence theoverall outcome that EBTs outperformed UC.This meta-analysis allows psychologists tomake the claim with more confidence for ageneral psychological principle regarding psychotherapy:Evidence-based treatments providebetter outcomes for youth than usual care.Confirming What the Results Reveal• Researchers use inferential statistics to determine whether an independentvariable has a reliable effect on a dependent variable.• Two methods to make inferences based on sample data are null hypothesistesting and confidence intervals.• Researchers use null hypothesis testing to determine whether meandifferences among groups in an experiment are greater than the differencesthat are expected simply because of error variation.• A statistically significant outcome is one that has a small likelihood ofoccurring if the null hypothesis were true.• Researchers determine whether an independent variable has had an effecton behavior by examining whether the confidence intervals for different