statisticalrethinkin..

8.2. MARKOV CHAIN MONTE CARLO 247forward planning or backwards record keeping, King Markov can satisfy his royal obligationto visit his people proportionally.8.2. Markov chain Monte Carloe precise algorithm King Markov used is a special case of the general METROPOLISALGORITHM from the real world. 98 And this algorithm is an example of Markov chain MonteCarlo. In real applications, the goal is of course not to help an autocrat schedule his journeys,but instead to draw samples from an unknown and usually complex target distribution, likea posterior probability density.• e “islands” in our objective are parameter values, and they need not be discrete,but can instead take on a continuous range of values as usual.• e “population sizes” in our objective are the posterior probabilities at each parametervalue.• e “weeks” in our objective are samples taken from the joint posterior of the parametersin the model.Provided the way we choose our proposed parameter values at each step is symmetric—sothat there is an equal chance of proposing from A to B and from B to A—then the Metropolisalgorithm will eventually give us a collection of samples from the joint posterior. We can thenuse these samples just like all the samples you’ve already used in this book.e Metropolis algorithm is the grandparent of several different strategies for gettingsamples from unknown posterior distributions. In the remainder of this section, I brieflyexplain the concepts behind two of the most important in contemporary Bayesian inference:Gibbs sampling and Hamiltonian (aka Hybrid, aka HMC) Monte Carlo. en we’ll turn tousing Hamiltonian Monte Carlo to do some new things with linear regression models.8.2.1. Gibbs sampling. e Metropolis algorithm works whenever the probability of proposinga jump to B from A is equal to the probability of proposing A from B, when the proposaldistribution is symmetric. ere is a more general method, known as Metropolis-Hastings 99 ,that allows asymmetric proposals. is would mean, in the context of King Markov’s fable,that the King’s coin were biased to lead him clockwise on average.Why would we want an algorithm that allows asymmetric proposals? One reason is thatit makes it easier to handle parameters, like standard deviations, that have boundaries atzero. A better reason, however, is that it allows us to generate savvy proposals that explorethe posterior density more efficiently. By “more efficiently,” I mean that we can acquire anequally good image of the posterior distribution in fewer steps.e most common way to generate savvy proposals is a technique known as GIBBS SAM-PLING. 100 Gibbs sampling is a variant of the Metropolis-Hastings algorithm that uses cleverproposals and is therefore more efficient. By “efficient,” I mean that you can get a goodestimate of the posterior from Gibbs sampling with many fewer samples than a comparableMetropolis approach. e improvement arises from adaptive proposals in which the distributionof proposed parameter values adjusts itself intelligently, depending upon the parametervalues at the moment.First, let’s describe the problem a little better. en I’ll explain how Gibbs sampling worksin detail. A major problem with basic Metropolis-Hastings (M-H), which you can think ofas King Markov’s system, is that it can take it a long time to adequately explore the posteriordensity. In principle, King Markov fulfills his promise to his people, but he might be deadbefore he visits the smaller islands again. e primary reason it can take so long to get around