Contents

208 PART III: Experimental Methodsyou will see, statistical inference is indirect (see, for example, Box 12.1 inChapter 12).Second, the data you have collected represent samples from a population;but in a sense, it is populations, not samples, that really matter. (If only samplemeans mattered, then you could simply look at the sample means to see ifthey were different.) The mean performance for the samples in the variousconditions of your experiment provides estimates that are used to infer themean of the population. When you make statements of statistical inference,you are using the sample means to make decisions (inferences) about differencesbetween (or among) population means. Once again we refer you toChapter 12 for a more complete discussion of these issues.Key ConceptKey ConceptNull Hypothesis Significance Testing (NHST) Researchers most frequently usenull hypothesis significance testing (NHST) to decide whether an independentvariable has produced an effect in an experiment. Null hypothesis significancetesting begins with the assumption that the independent variable has hadno effect. If we assume that the null hypothesis is true, we can use probabilitytheory to determine the probability that the difference we did observe in ourexperiment would occur “by chance.” A statistically significant outcome is onethat has only a small likelihood of occurring if the null hypothesis were true. A statisticallysignificant outcome means only that the difference we obtained in our experimentis larger than would be expected if error variation alone (i.e., chance)were responsible for the outcome.The outcome of an experiment is usually expressed in terms of the differencesbetween the means for the conditions in the experiment. How do weknow the probability of the obtained outcome in an experiment? Most often,researchers use inferential statistics tests such as the t-test or F-test. The t-testis used when there are two levels of the independent variable, and the F-testis used when there are three or more levels of the independent variable. Eachvalue of a t- or F-test has a probability value associated with it when the nullhypothesis is assumed. This probability can be determined once the researcherhas computed the value of the test statistic.Assuming the null hypothesis is true, just how small does the probabilityof our outcome need to be in order to be statistically significant? Scientiststend to agree that outcomes with probabilities (p) of less than 5 times out of100 (or p .05) are judged to be statistically significant. The probability valueresearchers use to decide that an outcome is statistically significant is calledthe level of significance. The level of significance is indicated by the Greek letteralpha ().We can now illustrate the procedures of null hypothesis testing to analyzethe video-game experiment we described earlier (see Table 6.1, p. 204). The firstresearch question we would ask is whether there was any overall effect of theindependent variable of video-game version. That is, did aggressive cognitiondiffer as a function of the three versions of the video game? The null hypothesisfor this overall test is that there is no difference among the population meansrepresented by the means of the experimental conditions (remember that the