Contents

394 PART V: Analyzing and Reporting ResearchTask Force on Statistical Inference (Wilkinson et al., 1999), use the simplest possibleanalysis. Second, when using NHST be sure to understand its limitationsand what the outcome of NHST allows you to say. Always consider reporting ameasure of effect magnitude when using NHST, and also a measure of power,especially when a nonsignificant result is found. Although there will be somesituations when effect size information is not warranted—for example, whentesting a theoretical prediction of direction only (e.g., Chow, 1988), these situationsare relatively rare. In many research situations, and in nearly all appliedsituations, effect size information is an important, even necessary, complementto NHST. Finally, researchers must “break the habit” of relying solely on NHSTand consider reporting confidence intervals for effect sizes in addition to, or evenrather than, p values associated with results of inferential tests. The APA PublicationManual (2010, p. 33) strongly recommends the use of confidence intervals.REPORTING RESULTS WHEN COMPARING TWO MEANSWe are now in a position to model a statement of results that takes into accountthe information gained from all three stages of data analysis, the complementaryevidence obtained by using confidence intervals (Chapter 11) andNHST, and the recommendations of the APA Publication Manual (2010) regardingreporting results (see especially pp. 32–35 of the Manual). Additional helpon reporting results using both NHST and a confidence interval (abbreviatedCI in a Results section) is found in Chapter 13.Reporting Results of the Vocabulary StudyWe may report the results as follows:The mean performance on the multiple-choice vocabulary test for collegestudents was 45.58 (SD 10.46); the mean of the older group was64.04 (SD 12.27). This difference was statistically significant, t(50) 5.84,p .001, d 1.65, 95% CI [12.11, 24.81]. Older participants in this study hada greater vocabulary size than did the younger participants.Commentary Descriptive statistics in the forms of means and standard deviationssummarize “what happened” in the experiment as a function of the independentvariable (age). Because the exact probability was less than .001, results are reportedat p .001, but note that exact probabilities are to be reported when .001or greater. The exact probability conveys information about the probability ofan exact replication (Posavac, 2002). That is, we know that the results are “morereliable” than if a larger exact p value had been obtained. This information isnot learned when only confidence intervals are reported. The sentence beginning“Older participants in this study . . .” summarizes in words what the statisticalanalysis revealed. It is always important to tell your reader directly what theanalysis shows. This becomes increasingly important as the number and complexityof analyses performed and reported in a research study increase. An effectsize (i.e., d) is also reported as recommended by the APA Publication Manual.This information is valuable to researchers doing meta-analyses and who wish tocompare results of studies using similar variables. On the other hand, confidence