statisticalrethinkin..

2.4. MAKING THE MODEL GO 47other probability concepts restrict probabilities to frequencies of data, real or imagined. Bayesianprobability includes frequencies as a special case.Inference under any probability concept will eventually make use of Bayes’ theorem. Commonintroductory examples of “Bayesian” analysis using HIV and DNA testing are not necessarilyBayesian, because since all of the elements of the calculation are frequencies of observations, a non-Bayesian analysis would do exactly the same thing. Instead, Bayesian approaches get to use Bayes’theorem more generally, to quantify uncertainty about theoretical entities that cannot be observed,like parameters and models. Powerful inferences can be produced under both Bayesian and non-Bayesian probability concepts, but different justifications and sacrifices are necessary.2.4. Making the model goRecall that your Bayesian model is a machine, a figurative golem. It has built-in definitionsfor the likelihood, the parameters, and the prior. And then at its heart lies a motor thatprocesses data, producing a posterior distribution. e action of this motor can be thoughtof as conditioning the prior on the data. As explained in the previous section, this conditioningis governed by the rules of probability theory, which defines a uniquely logical posteriorfor every prior, likelihood, and data.However, knowing the mathematical rule is oen of little help, because many of the interestingmodels in contemporary science cannot be conditioned formally, no matter yourskill in mathematics. And while some broadly useful models like linear regression can beconditioned formally, this is only possible if you constrain your choice of prior to specialforms that are easy to do mathematics with. We’d like to avoid forced modeling choices ofthis kind, instead favoring conditioning engines that can accommodate whichever prior ismost useful for inference.What this means is that various numerical techniques are needed to approximate themathematics that follows from the definition of Bayes’ theorem. In this book, you’ll meetthree different conditioning engines, numerical techniques for computing posterior distributions.(1) Grid approximation(2) Quadratic approximation(3) Markov chain Monte Carlo (MCMC)ere are many other engines, and new ones are being invented all the time. But the threeyou’ll get to know here are common and widely useful. In addition, as you learn them, you’llalso learn principles that will help you understand other techniques.Rethinking: How you fit the model is part of the model. Earlier in this chapter, I implicitly definedthe model as a composite of a prior and a likelihood. at definition is typical. But in practical terms,we should also consider how the model is fit to data as part of the model. In very simple problems,like the globe tossing example that consumes this chapter, calculation of the posterior density is trivialand foolproof. In even moderately complex problems, however, the details of fitting the model todata force us to recognize that our numerical technique influences our inferences. is is becausedifferent mistakes and compromises arise under different techniques. e same model fit to the samedata using different techniques may produce different answers. When something goes wrong, everypiece of the machine may be suspect. And so our golems carry with them their updating engines, asmuch slaves to their engineering as they are to the priors and likelihoods we program into them.