Information Theory, Inference, and Learning ... - Inference Group

Copyright Cambridge University Press 2003. On-screen viewing permitted. Printing not permitted. http://www.cambridge.org/0521642981You can buy this book for 30 pounds or $50. See http://www.inference.phy.cam.ac.uk/mackay/itila/ for links.54 3 — More about InferenceF Data (F a , F b )P (H 1 | s, F )P (H 0 | s, F )6 (5, 1) 222.26 (3, 3) 2.676 (2, 4) 0.71 = 1/1.46 (1, 5) 0.356 = 1/2.86 (0, 6) 0.427 = 1/2.320 (10, 10) 96.520 (3, 17) 0.2 = 1/520 (0, 20) 1.83Table 3.5. Outcome of modelcomparison between models H 1and H 0 for the ‘bent coin’. ModelH 0 states that p a = 1/6, p b = 5/6.H 0 is truep a = 1/681000/164100/12010/11/1-21/10-41/1000 50 100 150 20081000/164100/1210/101/1-21/10-41/1000 50 100 150 20081000/164100/1210/101/1-21/10-41/1000 50 100 150 200p a = 0.25H 1 is true81000/164100/12010/11/1-21/10-41/1000 50 100 150 20081000/164100/1210/101/1-21/10-41/1000 50 100 150 20081000/164100/1210/101/1-21/10-41/1000 50 100 150 20086p a = 0.51000/14100/1210/101/1-21/10-41/1000 50 100 150 20081000/164100/1210/101/1-21/10-41/1000 50 100 150 20081000/164100/1210/101/1-21/10-41/1000 50 100 150 200Figure 3.6. Typical behaviour ofthe evidence in favour of H 1 asbent coin tosses accumulate underthree different conditions(columns 1, 2, 3). Horizontal axisis the number of tosses, F . Thevertical axis on the left islnP (s | F, H1)P (s | F, H 0); the right-handvertical axis shows the values ofP (s | F, H 1)P (s | F, H 0) .The three rows show independentsimulated experiments.(See also figure 3.8, p.60.)⊲ Exercise 3.6. [2 ] Show that after F tosses have taken place, the biggest valuethat the log evidence ratiolog P (s | F, H 1)P (s | F, H 0 )(3.23)can have scales linearly with F if H 1 is more probable, but the logevidence in favour of H 0 can grow at most as log F .⊲ Exercise 3.7. [3, p.60] Putting your sampling theory hat on, assuming F a hasnot yet been measured, compute a plausible range that the log evidenceratio might lie in, as a function of F and the true value of p a , and sketchit as a function of F for p a = p 0 = 1/6, p a = 0.25, and p a = 1/2. [Hint:sketch the log evidence as a function of the random variable F a and workout the mean and standard deviation of F a .]Typical behaviour of the evidenceFigure 3.6 shows the log evidence ratio as a function of the number of tosses,F , in a number of simulated experiments. In the left-hand experiments, H 0was true. In the right-hand ones, H 1 was true, and the value of p a was either0.25 or 0.5.We will discuss model comparison more in a later chapter.