Essentials

108 ESSENTIALS OF STATISTICS# pairs k(k 1) (5.8)2where k is the number of groups. For the days of the week example the numberof possible t tests is (7 6)/2 42/2 21. If each of the possible 21 t tests is performedwith an alpha of .05, there is a good chance that at least one of the pairwisecomparisons will be significant, because an alpha of .05 implies that, on theaverage, one out of every 20 tests of a true null will yield significant results, andwe are dealing with 21 such tests (assuming that IQ is not affected by one’s day ofbirth).The probability of making one or more Type I errors in a group of tests thatare all part of the same experiment is called the experiment-wise alpha, symbolizedas EW. In contrast, the alpha used for each particular test is called the alpha percomparison, or pc. For the days/IQ example, pcwas .05, but EWwas larger than.5. (For c independent comparisons, EW 1 – (1 – pc) c ; the 21 t tests are not allmutually independent, but the formula gives a reasonable approximation in thiscase.) Clearly there is a need to control Type I errors, not just for individual t tests,but for whole experiments; an EWof .5 or more is just unacceptably large. Thisis where the one-way ANOVA comes in. If all multigroup experiments must producea significant F before t tests can be performed, then only .05 (or whateveralpha is used) of null experiments (like the days/IQ example) will reach significanceand be followed by t tests; 95% of null experiments will fail the ANOVAtest and not be followed up.The two-step system just described, in which the significance of the ANOVAdetermines whether you can perform all the possible t tests, was created byFisher. He called the follow-up t tests protected t tests, because the researcher wasprotected from testing an entire series of pairwise null hypotheses; a significantANOVA virtually guaranteed that there was at least one pair of conditions forwhich H 0was not true. Moreover, Fisher reasoned that when HOV could be assumed,the error term from the original ANOVA (i.e., MS W) could be used inplace of s 2 pooledfor every t test following the ANOVA, thus providing more df anda lower critical value for each test. (If HOV cannot be assumed for the ANOVA,each follow-up t test should be a separate-variances t test.) The formula forFisher’s protected t tests isX i X jt (5.9)MSW 1 1 n n j i