Contents

CHAPTER 12: Data Analysis and Interpretation: Part II. Tests of Statistical Significance and the Analysis Story 413a significant simple main effect at one of these levels, then one can conclude thatthere is a difference among the means (i.e., among the three means at that level ofvariable A). If that is the case, then the next step is to conduct comparisons of twomeans to analyze the simple main effect more fully. Comparisons of two meanswill help determine the nature of the differences among the levels. The statisticalanalysis for comparison between two means makes use of the t-test as describedearlier in this chapter. The MSerror from the omnibus 2 3 ANOVA SummaryTable is used in the t formula and the df associated with that term (24 in ourexample) are used to find the critical t value at the .05 level.If you are carrying out a simple main effects analysis for just two levels ofan independent variable, such as comparing mean performance at a 1 and a 2 forthe three levels of B, then you may use a t-test as you would for a two-meancomparison. Note that the sample sizes for your two-group t-test are based onthe number of participants in each of the two cells that you are contrasting. Inour hypothetical experiment n 5 for each group. Finally, as we did with thetwo-mean comparisons discussed above, you may again use the MSerror fromthe 2 3 ANOVA as the error term for your t-test. Degrees of freedom for thistwo-group t-test will be that associated with the MSerror for your omnibusANOVA. With two levels, a simple main effect compares the differencebetween two means and no additional comparisons are necessary.Once an interaction effect has been thoroughly analyzed, researchers canalso examine the main effect of each independent variable. In general, however,main effects are less interesting when an interaction effect is statisticallysignificant.Analysis with No Interaction Effect• If an omnibus analysis of variance indicates the interaction effect betweenindep en dent variables is not statistically significant, the next step isto determine whether the main effects of the variables are statisticallysignificant.• The source of a statistically significant main effect can be specified moreprecisely by performing comparisons that compare means two at a timeand by constructing confidence intervals.When the interaction effect is not statistically significant, the next step is toexamine the main effects of each independent variable. If the overall main effectfor an independent variable is not statistically significant, then there is nothingmore to do. However, if a main effect is statistically significant, there are severalapproaches a researcher may take. For example, if there are three or more levelsof the independent variable, the source of a statistically significant main effectcan be specified more precisely by performing comparisons of two means usingt-tests. Once again another approach is to construct confidence intervals aroundthe group means as we illustrated in Chapter 11 when analyzing a single-factorindependent groups design. The difference for the complex design is that thedata for one independent variable are collapsed across the levels of other independentvariables.