Information Theory, Inference, and Learning ... - Inference Group

Copyright Cambridge University Press 2003. On-screen viewing permitted. Printing not permitted. http://www.cambridge.org/0521642981You can buy this book for 30 pounds or $50. See http://www.inference.phy.cam.ac.uk/mackay/itila/ for links.49.5: Generalized parity-check matrices 585G T = H =Figure 49.4. The generatormatrix, parity-check matrix, and ageneralized parity-check matrix ofa repetition code with rate 1/ 3.{A, p} =check matrices are, as in (MacKay, 1999b; MacKay et al., 1998):• A diagonal line in a square indicates that that part of the matrix containsan identity matrix.• Two or more parallel diagonal lines indicate a band-diagonal matrix witha corresponding number of 1s per row.• A horizontal ellipse with an arrow on it indicates that the correspondingcolumns in a block are randomly permuted.• A vertical ellipse with an arrow on it indicates that the correspondingrows in a block are randomly permuted.• An integer surrounded by a circle represents that number of superposedrandom permutation matrices.Definition. A generalized parity-check matrix is a pair {A, p}, where A is abinary matrix and p is a list of the punctured bits. The matrix defines a setof valid vectors x, satisfyingAx = 0; (49.2)for each valid vector there is a codeword t(x) that is obtained by puncturingfrom x the bits indicated by p. For any one code there are many generalizedparity-check matrices.The rate of a code with generalized parity-check matrix {A, p} can beestimated as follows. If A is L × M ′ , and p punctures S bits and selects Nbits for transmission (L = N + S), then the effective number of constraints onthe codeword, M, isM = M ′ − S, (49.3)the number of source bits isand the rate is greater than or equal toK = N − M = L − M ′ , (49.4)R = 1 − M N = 1 − M ′ − SL − S . (49.5)