Information Theory, Inference, and Learning ... - Inference Group

Copyright Cambridge University Press 2003. On-screen viewing permitted. Printing not permitted. http://www.cambridge.org/0521642981You can buy this book for 30 pounds or $50. See http://www.inference.phy.cam.ac.uk/mackay/itila/ for links.29.10: Summary 381(1)(2)(3)A typical Markov chain Monte Carlo experiment involves an initial periodin which control parameters of the simulation such as step sizes may beadjusted. This is followed by a ‘burn in’ period during which we hope thesimulation ‘converges’ to the desired distribution. Finally, as the simulationcontinues, we record the state vector occasionally so as to create a list of states{x (r) } R r=1 that we hope are roughly independent samples from P (x).There are several possible strategies (figure 29.19):Figure 29.19. Three possibleMarkov chain Monte Carlostrategies for obtaining twelvesamples in a fixed amount ofcomputer time. Time isrepresented by horizontal lines;samples by white circles. (1) Asingle run consisting of one long‘burn in’ period followed by asampling period. (2) Fourmedium-length runs with differentinitial conditions and amedium-length burn in period.(3) Twelve short runs.1. Make one long run, obtaining all R samples from it.2. Make a few medium-length runs with different initial conditions, obtainingsome samples from each.3. Make R short runs, each starting from a different random initial condition,with the only state that is recorded being the final state of eachsimulation.The first strategy has the best chance of attaining ‘convergence’. The laststrategy may have the advantage that the correlations between the recordedsamples are smaller. The middle path is popular with Markov chain MonteCarlo experts (Gilks et al., 1996) because it avoids the inefficiency of discardingburn-in iterations in many runs, while still allowing one to detect problemswith lack of convergence that would not be apparent from a single run.Finally, I should emphasize that there is no need to make the points inthe estimate nearly-independent. Averaging over dependent points is fine – itwon’t lead to any bias in the estimates. For example, when you use strategy1 or 2, you may, if you wish, include all the points between the first and lastsample in each run. Of course, estimating the accuracy of the estimate isharder when the points are dependent.29.10 Summary• Monte Carlo methods are a powerful tool that allow one to sample fromany probability distribution that can be expressed in the form P (x) =1Z P ∗ (x).• Monte Carlo methods can answer virtually any query related to P (x) byputting the query in the form∫φ(x)P (x) ≃ 1 ∑φ(x (r) ). (29.49)Rr