Information Theory, Inference, and Learning ... - Inference Group

Copyright Cambridge University Press 2003. On-screen viewing permitted. Printing not permitted. http://www.cambridge.org/0521642981You can buy this book for 30 pounds or $50. See http://www.inference.phy.cam.ac.uk/mackay/itila/ for links.46.1: Traditional image reconstruction methods 551Differentiating with respect to W, and introducing F ≡ 〈 f j ′f j〉(cf. σ2fC −1 inthe Bayesian derivation above), we find that the optimal linear filter is:W opt = FR T [ RFR T + σ 2 ν I] −1. (46.12)If we identify F = σ 2 f C−1 , we obtain the optimal linear filter (46.8) of theBayesian derivation. The ad hoc assumptions made in this derivation were thechoice of a quadratic error measure, and the decision to use a linear estimator.It is interesting that without explicit assumptions of Gaussian distributions,this derivation has reproduced the same estimator as the Bayesian posteriormode, f MP .The advantage of a Bayesian approach is that we can criticize these assumptionsand modify them in order to make better reconstructions.Other image modelsThe better matched our model of images P (f | H) is to the real world, the betterour image reconstructions will be, and the less data we will need to answerany given question. The Gaussian models which lead to the optimal linearfilter are spectacularly poorly matched to the real world. For example, theGaussian prior (46.3) fails to specify that all pixel intensities in an image arepositive. This omission leads to the most pronounced artefacts where the imageunder observation has high contrast or large black patches. Optimal linearfilters applied to astronomical data give reconstructions with negative areas inthem, corresponding to patches of sky that suck energy out of telescopes! Themaximum entropy model for image deconvolution (Gull and Daniell, 1978)was a great success principally because this model forced the reconstructedimage to be positive. The spurious negative areas and complementary spuriouspositive areas are eliminated, and the quality of the reconstruction isgreatly enhanced.The ‘classic maximum entropy’ model assigns an entropic priorP (f | α, m, H Classic ) = exp(αS(f, m))/Z, (46.13)whereS(f, m) = ∑ i(f i ln(m i /f i ) + f i − m i ) (46.14)(Skilling, 1989). This model enforces positivity; the parameter α defines acharacteristic dynamic range by which the pixel values are expected to differfrom the default image m.The ‘intrinsic-correlation-function maximum-entropy’ model (Gull, 1989)introduces an expectation of spatial correlations into the prior on f by writingf = Gh, where G is a convolution with an intrinsic correlation function, andputting a classic maxent prior on the underlying hidden image h.Probabilistic moviesHaving found not only the most probable image f MP but also error bars onit, Σ f|d , one task is to visualize those error bars. Whether or not we useMonte Carlo methods to infer f, a correlated random walk around the posteriordistribution can be used to visualize the uncertainties and correlations. Fora Gaussian posterior distribution, we can create a correlated sequence of unitnormal random vectors n usingn (t+1) = cn (t) + sz, (46.15)