Contents

392 PART V: Analyzing and Reporting ResearchThe numerator of the repeated measures t is the mean of the differencescores ( __ D ) and is algebraically equivalent to the difference between the samplemeans (i.e., __ X 1 __ X 2 ). The denominator is the estimated standard error ofthe difference scores (see Chapter 11). Statistical significance is determinedby comparing the obtained t with critical values of t with df equal to N 1.In this case, N refers to the number of participants or pairs of scores in theexperiment. You interpret the obtained t as you would the t obtained in anindependent groups design.As noted in Chapter 11, assessing effect size in a matched groups or repeatedmeasures design is somewhat more complex than for an independent groupsdesign (see Cohen, 1988, and Rosenthal & Rosnow, 1991, for information pertainingto the calculation of d in these cases).STATISTICAL SIGNIFICANCE AND SCIENTIFICOR PRACTICAL SIGNIFICANCE• We must recognize the fact that statistical significance is not the same assci entific significance.• We also must acknowledge that statistical significance is not the same aspractical or clinical significance.Tests of statistical significance are an important tool in the analysis of researchfindings. We must be careful, however, to interpret statistically significantfindings correctly (see Box 12.3). We must also be careful not to confuse astatistically significant finding with a scientifically significant finding. Whetherthe results of a study are important to the scientific community will dependon the nature of the variable under study (the effects of some variables aresimply more important than those of others), how sound the study is (statisticallysignificant findings can be produced with poorly done studies), and othercriteria such as effect size (see, for example, Abelson, 1995).Similarly, the practical or clinical significance of a treatment effect dependson factors other than statistical significance. These include the external validityassociated with the study, the size of the effect, and various practical considerations(including financial ones) associated with a treatment’s implementation.Even a statistically significant outcome showing a large effect size is not a guaranteeof its practical or clinical significance. A very large effect size might beobtained as a part of a study that does not generalize well from the laboratoryto the real world (i.e., has low external validity); thus, the results may be of littlevalue to the applied psychologist. Moreover, a relatively large treatment effectthat does generalize well to real-world settings may never be applied becauseit is too costly, too difficult to implement, too controversial, or too similar in itseffects to existing treatments.It is also possible that, given enough power, a small effect size will be statisticallysignificant. Small effect sizes may not be practically important outsidethe laboratory. As we described in Chapter 6, external validity is an empiricalquestion. It is important to conduct a study under conditions similar to those inwhich the treatment will be used in order to see whether a finding is practically