Contents

CHAPTER 12: Data Analysis and Interpretation: Part II. Tests of Statistical Significance and the Analysis Story 391Key ConceptIndependent GroupsRecall that a study was conducted in which the vocabulary size of college studentsand older adults was assessed. The appropriate inferential test for thissituation is a t-test for independent groups. We may use this test to evaluatethe difference between the mean percent multiple-choice performance ofthe college and older adult samples. Statistical software programs typicallyprovide the actual probability of an obtained t as part of the output. In fact, theAPA Publication Manual (2010) advises that the exact probability be reported.When the exact probability is less than .001 (e.g., p .0004), statistical softwareprograms frequently report the exact probability as .000. (This was the case forthe analysis reported above.) Of course, the exact probability is not .000 butsomething less than .001.Therefore, for the vocabulary study we have been discussing, the result ofthe inferential statistics test can be summarized ast(50) 5.84, p .001In Chapter 11 we showed how an effect size, d, can be calculated for a comparisonbetween two means. A measure of effect size should always be reportedwhen NHST is used. You may recall that in Chapter 11 we calculated d for thevocabulary study as 1.65. Cohen’s d also can be calculated from the outcome ofthe independent groups t-test according to the following formula:That is,d ____ 2t df(see Rosenthal & Rosnow, 1991)d _______ 2(5.84)50 11.68 _____7.07 1.65Repeated Measures DesignsThus far we have considered experiments involving two independent groupsof subjects. As you are aware, experiments can also be carried out by havingeach subject participate in each condition of the experiment or by “matching”subjects on some measure related to the dependent variable (e.g., IQ scores,weight). Such experiments are called matched groups (see Chapter 6), withinsubjectsdesigns, or repeated measures designs (see Chapter 7). The logic ofNHST is the same in a repeated measures design as it is in an independentgroups design. However, the t-test comparing two means takes on a differentform in a repeated measures design. The t-test in this situation is typicallyKey Concept called a direct-difference t or repeated measures (within-subjects) t-test. Whencarrying out a computer-assisted analysis when subjects are in both conditionsof the experiment you will find that the data are entered differently than whenindependent groups of subjects are tested.