Information Theory, Inference, and Learning ... - Inference Group

Copyright Cambridge University Press 2003. On-screen viewing permitted. Printing not permitted. http://www.cambridge.org/0521642981You can buy this book for 30 pounds or $50. See http://www.inference.phy.cam.ac.uk/mackay/itila/ for links.viiiPreface1 Introduction to Information Theory2 Probability, Entropy, and Inference3 More about InferenceI Data Compression4 The Source Coding Theorem5 Symbol Codes6 Stream Codes7 Codes for IntegersII Noisy-Channel Coding8 Dependent Random Variables9 Communication over a Noisy Channel10 The Noisy-Channel Coding Theorem11 Error-Correcting Codes and Real ChannelsIII Further Topics in Information Theory12 Hash Codes13 Binary Codes14 Very Good Linear Codes Exist15 Further Exercises on Information Theory16 Message Passing17 Constrained Noiseless Channels18 Crosswords and Codebreaking19 Why have Sex?IV Probabilities and Inference20 An Example Inference Task: Clustering21 Exact Inference by Complete Enumeration22 Maximum Likelihood and Clustering23 Useful Probability Distributions24 Exact Marginalization25 Exact Marginalization in Trellises26 Exact Marginalization in Graphs27 Laplace’s Method28 Model Comparison and Occam’s Razor29 Monte Carlo Methods30 Efficient Monte Carlo Methods31 Ising Models32 Exact Monte Carlo Sampling33 Variational Methods34 Independent Component Analysis35 Random Inference Topics36 Decision Theory37 Bayesian Inference and Sampling TheoryV Neural networks38 Introduction to Neural Networks39 The Single Neuron as a Classifier40 Capacity of a Single Neuron41 Learning as Inference42 Hopfield Networks43 Boltzmann Machines44 Supervised Learning in Multilayer Networks45 Gaussian Processes46 DeconvolutionVISparse Graph CodesShort Course onInformation Theory47 Low-Density Parity-Check Codes48 Convolutional Codes and Turbo Codes49 Repeat–Accumulate Codes50 Digital Fountain Codes