Contents

406 PART V: Analyzing and Reporting ResearchOne comparison of two means we could make for the memory experiment isto compare the mean performance for the memory-training groups (combined)and the control group. The mean retention for the three memory training groupsis 13.67 (n 15), and the mean for the control group is 10.00 (n 5). We can ask,does memory training, regardless of type (i.e., story, imagery, rhymes), lead tobetter memory retention than no memory training (control)? The null hypothesisis that the two population means do not differ (and the sample means differ bychance alone). When the appropriate values are substituted into the formula fort given above, we observe a statistically significant effect, t(16) 4.66, p .0003.Thus, memory training in this experiment, regardless of type, resulted in bettermemory retention for the words compared to no training. You can see that thisstatement is more specific than the statement we could make based on the omnibusF test, in which we could say only that the variation across the four conditionsof the experiment was larger than that expected based on chance alone.Cohen’s d may be calculated for comparisons of two means using the resultsof the t-test. The formula for Cohen’s d in this situation is2(t)d _______ df errorFor the comparison between the three memory-training groups and the controlgroup, substituting the value of 4.66 into the formula and with 16 df error , theeffect size, d, is 2.33. According to Cohen’s criteria for effect sizes, this can beinterpreted as a large effect of memory instruction relative to no instruction.When using a t-test we are seeking to make a decision about rejecting or notrejecting the null hypothesis with a specific probability (e.g., p .05). As notedpreviously, the exact probability associated with the outcome of NHST can beimportant when interpreting results (e.g., Posavac, 2002). The lower the exactprobability, the greater is the likelihood that an exact replication would permitrejecting the null hypothesis at p .05 (see Zechmeister & Posavac, 2003).Minimally, we want to report the lowest probability for statistical significancefor which we have information. (Computers automatically give the exact probabilityof our test result.)The results of the t comparison also permit us to contrast results with previousstudies in two ways. First, we can note whether our experiment’s findingsfor statistical significance are similar to those observed in a previous experiment.That is, did we replicate a statistically significant finding? Second, wecan calculate an effect size (e.g., Cohen’s d) for this two-mean comparison thatmay be compared with effects obtained in previous experiments, perhaps aspart of a meta-analysis. Neither of these contrasts is easy to do using confidenceintervals. That is, unlike NHST, confidence intervals do not provide anexact probability associated with a difference seen in our experiment and thecalculation of an effect size is more directly carried out following a t-test (seeChapter 11).In summary, we encourage you to look at your data and differences betweenmeans using more than one statistical technique, seeking evidence for “whathappened” from different approaches to data analysis.