Information Theory, Inference, and Learning ... - Inference Group

Copyright Cambridge University Press 2003. On-screen viewing permitted. Printing not permitted. http://www.cambridge.org/0521642981You can buy this book for 30 pounds or $50. See http://www.inference.phy.cam.ac.uk/mackay/itila/ for links.23.4: Distributions over periodic variables 3152.521.510.500 1 2 310.10.80.70.60.50.40.30.20.100.1-4 -2 0 2 4Figure 23.5. Two inverse gammadistributions, with parameters(s, c) = (1, 3) (heavy lines) and10, 0.3 (light lines), shown onlinear vertical scales (top) andlogarithmic vertical scales(bottom); and shown as afunction of x on the left andl = ln x on the right.0.010.010.0010.0010.00010 1 2 30.0001-4 -2 0 2 4vln vGamma and inverse gamma distributions crop up in many inference problemsin which a positive quantity is inferred from data. Examples includeinferring the variance of Gaussian noise from some noise samples, and inferringthe rate parameter of a Poisson distribution from the count.Gamma distributions also arise naturally in the distributions of waitingtimes between Poisson-distributed events. Given a Poisson process with rateλ, the probability density of the arrival time x of the mth event isLog-normal distributionλ(λx) m−1(m−1)!e −λx . (23.22)Another distribution over a positive real number x is the log-normal distribution,which is the distribution that results when l = ln x has a normal distribution.We define m to be the median value of x, and s to be the standarddeviation of ln x.P (l | m, s) = 1 ()Z exp (l − ln m)2−2s 2 l ∈ (−∞, ∞), (23.23)whereimpliesP (x | m, s) = 1 ()1x Z exp (ln x − ln m)2−2s 2Z = √ 2πs 2 , (23.24)x ∈ (0, ∞). (23.25)0.40.350.30.250.20.150.10.0500 1 2 3 4 50.10.010.0010.00010 1 2 3 4 5Figure 23.6. Two log-normaldistributions, with parameters(m, s) = (3, 1.8) (heavy line) and(3, 0.7) (light line), shown onlinear vertical scales (top) andlogarithmic vertical scales(bottom). [Yes, they really dohave the same value of themedian, m = 3.]23.4 Distributions over periodic variablesA periodic variable θ is a real number ∈ [0, 2π] having the property that θ = 0and θ = 2π are equivalent.A distribution that plays for periodic variables the role played by the Gaussiandistribution for real variables is the Von Mises distribution:P (θ | µ, β) = 1 exp (β cos(θ − µ)) θ ∈ (0, 2π). (23.26)ZThe normalizing constant is Z = 2πI 0 (β), where I 0 (x) is a modified Besselfunction.