Information Theory, Inference, and Learning ... - Inference Group

Copyright Cambridge University Press 2003. On-screen viewing permitted. Printing not permitted. http://www.cambridge.org/0521642981You can buy this book for 30 pounds or $50. See http://www.inference.phy.cam.ac.uk/mackay/itila/ for links.Index 623fountain code, 589Gallager, 557Golay, 209good, 183, 184, 207, 214, 218Hamming, 19, 214in DNA replication, 280in protein synthesis, 280interleaving, 186linear, 9, 171, 183, 184, 229coding theorem, 229low-density generator-matrix,218, 590low-density parity-check, 20, 187,218, 557, 596fast encoding, 569profile, 569staircase, 569LT code, 590maximum distance separable, 220nonlinear, 187P 3, 218parity-check code, 220pentagonful, 221perfect, 208, 211, 212, 219, 589practical, 183, 187product code, 184, 214quantum, 572random, 184random linear, 211, 212raptor code, 594rate, 152, 229rateless, 590rectangular, 184Reed–Solomon code, 571, 589repeat–accumulate, 582repetition, 183simple parity, 218sparse graph, 556density evolution, 566syndrome decoding, 11, 371variable rate, 238, 590very bad, 207very good, 183weight enumerator, 206with varying level of protection,239error-reject curves, 533errors, see channelestimator, 48, 307, 320, 446, 459eugenics, 273euro, 63evidence, 29, 53, 298, 322, 347, 531typical behaviour of, 54, 60evolution, 269, 279as learning, 277Baldwin effect, 279colour vision, 554of the genetic code, 279evolutionary computing, 394, 395exact sampling, 413exchange rate, 601exchangeability, 263exclusive or, 590EXIT chart, 567expectation, 27, 35, 37expectation propagation, 340expectation–maximization algorithm,283, 432experimental design, 463experimental skill, 309explaining away, 293, 295exploit, 453explore, 453exponential distribution, 45, 313on integers, 311exponential-family, 307, 308expurgation, 167, 171extended channel, 153, 159extended code, 92extended ensemble, 76extra bit, 98, 101extreme value, 446eye movements, 554factor analysis, 437, 444factor graph, 334–336, 434, 556, 557,580, 583factorial, 2fading channel, 186feedback, 506, 589female, 277ferromagnetic, 400Feynman, Richard, 422Fibonacci, 253field, 605, see Galois fieldfile storage, 188finger, 119finite field theory, see Galois fieldfitness, 269, 279fixed point, 508Florida, 355fluctuation analysis, 446fluctuations, 401, 404, 427, 602focus, 529football pools, 209forensic, 47, 421forgery, 199, 200forward pass, 244forward probability, 27forward–backward algorithm, 326, 330Fotherington–Thomas, 241fountain code, 589Fourier transform, 88, 219, 339, 544,568fovea, 554free energy, 257, 407, 409, 410, seepartition functionminimization, 423variational, 423frequency, 26frequentist, 320, see sampling theoryFrey, Brendan J., 353Frobenius–Perron theorem, 410frustration, 406full probabilistic model, 156function minimization, 473functions, 246gain, 507Galileo code, 186Gallager code, 557Gallager, Robert G., 170, 187, 557Galois field, 185, 224, 567, 568, 605gambling, 455game, see puzzleBridge, 126chess, 451guess that tune, 204guessing, 110life, 520sixty-three, 70submarine, 71three doors, 57, 60, 454twenty questions, 70game show, 57, 454game-playing, 451gamma distribution, 313, 319gamma function, 598ganglion cells, 491Gaussian channel, 155, 177Gaussian distribution, 2, 36, 176, 312,321, 398, 549N–dimensional, 124approximation, 501parameters, 319sample from, 312Gaussian processes, 535variational classifier, 547general position, 484generalization, 483generalized parity-check matrix, 581generating function, 88generative model, 27, 156generator matrix, 9, 183genes, 201genetic algorithm, 269, 395, 396genetic code, 279genome, 201, 280geometric progression, 258geostatistics, 536, 548GF(q), see Galois fieldGibbs entropy, 85Gibbs sampling, 370, 391, 418, seeMonte Carlo methodsGibbs’ inequality, 34, 37, 44Gilbert–Varshamov conjecture, 212Gilbert–Varshamov distance, 212, 221Gilbert–Varshamov rate, 212Gilks, Wally R., 393girlie stuff, 58Glauber dynamics, 370Glavieux, A., 186Golay code, 209golden ratio, 253good, see error-correcting codeGood, Jack, 265gradient descent, 476, 479, 498, 529natural, 443graduated non-convexity, 518Graham, Ronald L., 175grain size, 180graph, 251factor graph, 334of code, 19, 20, 556graphs and cycles, 242guerilla, 242