Information Theory, Inference, and Learning ... - Inference Group

Copyright Cambridge University Press 2003. On-screen viewing permitted. Printing not permitted. http://www.cambridge.org/0521642981You can buy this book for 30 pounds or $50. See http://www.inference.phy.cam.ac.uk/mackay/itila/ for links.IndexΓ, 598Φ(z), 514χ 2 , 40, 323, 458, 459λ, 119σ N and σ N−1, 320:=, 600?, 4192s, 156Abu-Mostafa, Yaser, 482acceptance rate, 365, 369, 394acceptance ratio method, 379accumulator, 254, 570, 582activation function, 471activity rule, 470, 471adaptive direction sampling, 393adaptive models, 101adaptive rejection sampling, 370address, 201, 468Aiyer, Sree, 518Alberto, 56alchemists, 74algebraic coding theory, 19, 574algorithm, see learning algorithmsBCJR, 330belief propagation, 330, 336covariant, 442EM, 432exact sampling, 413expectation–maximization, 432function minimization, 473genetic, 395, 396Hamiltonian Monte Carlo, 387,496independent component analysis,443Langevin Monte Carlo, 496leapfrog, 389max–product, 339message-passing, 330min–sum, 339Monte Carlo, see Monte CarlomethodsNewton–Raphson, 303, 441perfect simulation, 413sum–product, 336Viterbi, 340Alice, 199Allais paradox, 454alphabetical ordering, 194America, 354American, 238, 260amino acid, 201, 204, 279, 362anagram, 200annealing, 379, 392, 397deterministic, 518importance sampling, 379antiferromagnetic, 400ape, 269approximationby Gaussian, 2, 301, 341, 350,496Laplace, 341, 547of complex distribution, 185, 282,364, 422, 433of density evolution, 567saddle-point, 341Stirling, 1variational, 422arabic, 127architecture, 470, 529arithmetic coding, 101, 110, 111decoder, 118software, 121uses beyond compression, 118,250, 255arithmetic progression, 344arms race, 278artificial intelligence, 121, 129associative memory, 468, 505, 507assumptions, 26, 50astronomy, 551asymptotic equipartition, 80, 384why it is a misleading term, 83Atlantic, 173AutoClass, 306automatic relevance determination,532, 544automobile data reception, 594average, 26, see expectationAWGN, 177background rate, 307backpropagation, 473, 475, 528, 535backward pass, 244bad, see error-correcting codeBalakrishnan, Sree, 518balance, 66Baldwin effect, 279ban (unit), 264Banburismus, 265band-limited signal, 178bandwidth, 178, 182bar-code, 262, 399base transitions, 373base-pairing, 280basis dependence, 306, 342bat, 213, 214battleships, 71Bayes’ theorem, 6, 24, 25, 27, 28,48–50, 53, 148, 324, 344,347, 446, 493, 522Bayes, Rev. Thomas, 51Bayesian belief networks, 293Bayesian inference, 26, 346, 457BCH codes, 13BCJR algorithm, 330, 578bearing, 307Belarusian, 238belief, 57belief propagation, 330, 557, seemessage passing andsum–product algorithmBenford’s law, 446bent coin, 1, 30, 38, 51, 76, 113, 307Berlekamp, Elwyn, 172, 213Bernoulli distribution, 117Berrou, C., 186bet, 200, 209, 455beta distribution, 316beta function, 316beta integral, 30Bethe free energy, 434Bhattacharyya parameter, 215bias, 345, 506in neural net, 471in statistics, 306, 307, 321biexponential, 88, 313, 448bifurcation, 89, 291, 426binary entropy function, 2, 15binary erasure channel, 148, 151binary images, 399binary representations, 132binary symmetric channel, 4, 148,151, 211, 215, 229binding DNA, 201binomial distribution, 1, 311bipartite graph, 19birthday, 156, 157, 160, 198, 200bit, 3, 73bit (unit), 264bits back, 104, 108, 353bivariate Gaussian, 388black, 355Bletchley Park, 265Blind Watchmaker, 269, 396block code, 9, see source code orerror-correcting codeblock-sorting, 121blood group, 55blow up, 306blur, 549620