2DkcTXceO

26How do we choose our default methods?Andrew GelmanDepartment of StatisticsColumbia University, New YorkThe field of statistics continues to be divided into competing schools ofthought. In theory one might imagine choosing the uniquely best methodfor each problem as it arises, but in practice we choose for ourselves (andrecommend to others) default principles, models, and methods to be used in awide variety of settings. This chapter briefly considers the informal criteria weuse to decide what methods to use and what principles to apply in statisticsproblems.26.1 Statistics: The science of defaultsApplied statistics is sometimes concerned with one-of-a-kind problems, butstatistical methods are typically intended to be used in routine practice. Thisis recognized in classical theory (where statistical properties are evaluatedbased on their long-run frequency distributions) and in Bayesian statistics(averaging over the prior distribution). In computer science, machine learningalgorithms are compared using cross-validation on benchmark corpuses,which is another sort of reference distribution. With good data, a classicalprocedure should be robust and have good statistical properties under a widerange of frequency distributions, Bayesian inferences should be reasonableeven if averaging over alternative choices of prior distribution, and the relativeperformance of machine learning algorithms should not depend stronglyon the choice of corpus.How do we, as statisticians, decide what default methods to use? Here I amusing the term “method” broadly, to include general approaches to statistics(e.g., Bayesian, likelihood-based, or nonparametric) as well as more specificchoices of models (e.g., linear regression, splines, or Gaussian processes) andoptions within a model or method (e.g., model averaging, L 1 regularization,or hierarchical partial pooling). There are so many choices that it is hard to293