Information Theory, Inference, and Learning ... - Inference Group

Copyright Cambridge University Press 2003. On-screen viewing permitted. Printing not permitted. http://www.cambridge.org/0521642981You can buy this book for 30 pounds or $50. See http://www.inference.phy.cam.ac.uk/mackay/itila/ for links.6.9: Solutions 131Figure 6.14. The 100-step time-history of a cellular automaton with 400 cells.The variance of r 2 , similarly, is N times the variance of x 2 , where x is aone-dimensional Gaussian variable.∫)var(x 2 1) = dx(2πσ 2 ) 1/2 x4 exp(− x22σ 2 − σ 4 . (6.26)The integral is found to be 3σ 4 (equation (6.14)), so var(x 2 ) = 2σ 4 . Thus thevariance of r 2 is 2Nσ 4 .For large N, the central-limit theorem indicates that r 2 has a Gaussiandistribution with mean Nσ 2 and standard deviation √ 2Nσ 2 , so the probabilitydensity of r must similarly be concentrated about r ≃ √ Nσ.The thickness of this shell is given by turning the standard deviationof r 2 into a standard deviation on r: for small δr/r, δ log r = δr/r =( 1/ 2)δ log r 2 = ( 1/ 2)δ(r 2 )/r 2 , so setting δ(r 2 ) = √ 2Nσ 2 , r has standard deviationδr = ( 1/ 2)rδ(r 2 )/r 2 = σ/ √ 2.The probability density of the Gaussian at a point x shell where r = √ NσisP (x shell ) =( )1(2πσ 2 ) N/2 exp − Nσ22σ 2 =Whereas the probability density at the origin isP (x = 0) =1(−(2πσ 2 ) N/2 exp N ). (6.27)21(2πσ 2 . (6.28))N/2Thus P (x shell )/P (x = 0) = exp (−N/2) . The probability density at the typicalradius is e −N/2 times smaller than the density at the origin. If N = 1000, thenthe probability density at the origin is e 500 times greater.