2DkcTXceO

52 Unorthodox journey to statisticsdividual differences are so important that it is essential to distinguish betweenmultiple methods applied to the same groups of individuals and methods appliedto different independent groups. I found that the statisticians were lesssensitive to this than the psychologists. In the consulting classes, they sometimesforgot to even ask about that. Also, they sometimes seemed to thinkthat with the addition of a few variables (e.g., gender, ethnicity, other distinguishingvariables) they could take care of individual differences, and treat thedata as conditionally independent observations, whereas psychologists wouldbe quite wary of making that assumption. There must be other fields in whichcareful experimental thinking is necessary. This is not statistics in a narrowsense, but is certainly important for applied statisticians who may be involvedin designing studies.5.3 Introduction to and work in multiplicityIn addition to teaching many psychology courses during my time at Kansas,I also taught most of the statistics courses to the psychology undergraduateand graduate students. Analysis of variance (ANOVA) was perhaps the mostwidely-used procedure in experimental psychology. Consider, for example, aone-way treatment layout to be analyzed as an ANOVA. Given a significantF statistic, students would then compare every treatment with every otherto see which were different using methods with a fixed, conventional Type Ierror rate α (usually .05) for each. I realized that the probability of somefalse conclusions among these comparisons would be well above this nominalType I error level, growing with the number of such comparisons. This piquedmy interest in multiplicity problems, which eventually became my major areaof research.The criterion most widely considered at that time was the family-wiseerror rate (FWER), the probability of one or more false rejections (i.e., rejectionsof true hypotheses) in a set of tests. If tests are carried out individuallywith specified maximum (Type I) error rates, the probability of one or moreerrors increases with the number of tests. Thus, the error rate for the wholeset should be considered. The statistical papers I read all referred to an unpublishedmanuscript, “The Problem of Multiple Comparisons,” by John W.Tukey (1953). In those days, before Xerox, it was impossible to get copies ofthat manuscript. It was frustrating to have to use secondary sources. Fortunately,with the advent of Xerox, that problem has disappeared, and now,in addition, the manuscript is included in Tukey’s Collected Works (Braun,1994).Tukey’s treatment was extremely insightful and organized the field forsome time to follow. In his introduction, he notes that he should have publishedit as a book at that time but “One reason this did not happen was the only