Contents

CHAPTER 12: Data Analysis and Interpretation: Part II. Tests of Statistical Significance and the Analysis Story 409BOX 12.4ESTIMATING ERROR AND SENSITIVITY IN A REPEATED MEASURES DESIGNOne distinctive characteristic of the analysis ofrepeated measures designs is the way in whicherror variation is estimated. We described earlierthat for the random groups design, individual differencesamong participants that are balancedacross groups provide the estimate of error variation,which becomes the denominator of theF-test. Because individuals participate in only onecondition in these designs, differences amongparticipants cannot be eliminated—they can onlybe balanced. In repeated measures designs,on the other hand, there is systematic variationamong participants. Some participants consistentlyperform better across conditions, and someparticipants consistently perform worse. Becauseeach individual participates in each condition ofrepeated measures designs, however, differencesamong participants contribute equally tothe mean performance in each condition. Accordingly,any differences among the means for eachcondition in repeated measures designs cannotbe the result of systematic differences amongparticipants. In repeated measures designs, however,differences among participants are not justbalanced—they are actually eliminated from theanalysis. The ability to eliminate systematic variationdue to participants in repeated measuresdesigns makes these designs generally more sensitivethan random groups designs.of a repeated measures analysis of variance would be done using a statistical softwarepackage on a computer. Our focus now is on interpreting the values in thesummary table and not on how these values are computed. Table 12.4 lists the foursources of variation in the analysis of a repeated measures design with one manipulatedindependent variable. Reading from the bottom of the summary tableup, these sources are (1) total variation, (2) residual variation, (3) variation due tointerval length (the independent variable), and (4) variation due to subjects.As in any summary table, the most critical pieces of information are the F-test forthe effect of the independent variable of interest and the probability associated withthat F-test assuming the null hypothesis is true. The important F-test in Table 12.4is the one for interval length. The numerator for this F-test is the mean square (MS)for interval length; the denominator is the residual MS. There are four intervallengths, so there are 3 degrees of freedom (df ) for the numerator. There are 12 df forthe residual variation. We can obtain the df for the residual variation by subtractingthe df for subjects and for interval length from the total df(19 4 3 12). Theobtained F of 15.6 has a probability under the null hypothesis of .0004, which isless than the .05 level of significance we have chosen as our criterion for statisticalsignificance. So we reject the null hypothesis and conclude that the interval lengthwas a source of systematic variation. This means that we can conclude that the participants’estimates did differ systematically as a function of interval length.Figure 12.2 shows 95% confidence intervals around the means in the timeperceptionexperiment. The procedure for constructing these intervals is thesame as that for the independent groups experiment. Intervals were constructedusing the MS error (residual) in the omnibus ANOVA (as recommended by Loftus& Masson, 1994). That is,95% CI __X __________(MS error /n) ( t crit )