Essentials

116 ESSENTIALS OF STATISTICSDON’T FORGETPlanned contrasts can be much morelikely to reach significance than an ordinaryone-way ANOVA (eventhough the critical value will be somewhathigher) if the pattern of meanswas predicted correctly. However,your chances of significance can bemuch lower if the pattern tested byyour contrast turns out not to matchyour data very well.ered a legitimate and often desirablealternative to performing a one-wayANOVA, assuming there is a strongtheoretical justification for yourchoice of contrasts. In some cases aresearcher may want to test severalcomplex (and possibly pairwise)comparisons that are not mutuallyorthogonal. Such testing is consideredreasonable if the desired EW(usually .05) is divided by the numberof planned comparisons, and thatfractional probability is used as the alpha for each comparison. If four comparisonsare planned then the p value for each (as given by statistical software) iscompared to .0125 (i.e., .05/4).Post Hoc Complex Comparisons: Scheffé’s TestIf a one-way ANOVA is not significant, it is often possible to find a reasonablecomplex comparison that will be significant, but if it was not literally planned inadvance the comparison had better be so obvious that it should have beenplanned. Otherwise, it will be hard to convince a journal editor (or reviewer) toaccept a test of such a comparison as a planned test. On the other hand, if a onewayANOVA is significant, it is considered reasonable to test particular complexcomparisons to specify the effect further, but one is required to use a post hoc testthat is so stringent that no complex comparison will reach significance if the omnibus(i.e., overall) ANOVA did not. This test is called the Scheffé test, and it isremarkably simple and flexible. The critical F for testing a post hoc complex comparisonis just k – 1 times the critical F for the omnibus ANOVA. According toScheffé’s test the critical F for testing any of the comparisons in our preceding exampleat the .05 level would be 6.7 (i.e., 2 3.35) if the comparison were notplanned.The largest F you can get when testing a comparison occurs when all ofSS betweenfits into one contrast. Then SS betweenis divided by 1 instead of k – 1, makingthe F for the contrast k – 1 times the F for the omnibus ANOVA. If the criticalvalue is also multiplied by k – 1, as in Scheffé’s test, the largest possible contrastwon’t be significant unless the omnibus ANOVA is significant. That is thelogic behind Scheffé’s test. Because it is so easy to use and stringent about keeping EWto the value set for it, Scheffé’s test has sometimes been used by re-