Information Theory, Inference, and Learning ... - Inference Group

Copyright Cambridge University Press 2003. On-screen viewing permitted. Printing not permitted. http://www.cambridge.org/0521642981You can buy this book for 30 pounds or $50. See http://www.inference.phy.cam.ac.uk/mackay/itila/ for links.13.10: Dual codes 217One way of thinking of this equation is that each row of H specifies a vectorto which t must be orthogonal if it is a codeword.The generator matrix specifies K vectors from which all codewordscan be built, and the parity-check matrix specifies a set of M vectorsto which all codewords are orthogonal.The dual of a code is obtained by exchanging the generator matrixand the parity-check matrix.Definition. The set of all vectors of length N that are orthogonal to all codewordsin a code, C, is called the dual of the code, C ⊥ .If t is orthogonal to h 1 and h 2 , then it is also orthogonal to h 3 ≡ h 1 + h 2 ;so all codewords are orthogonal to any linear combination of the M rows ofH. So the set of all linear combinations of the rows of the parity-check matrixis the dual code.For our Hamming (7, 4) code, the parity-check matrix is (from p.12):H = [ PI 3]=⎡⎣1 1 1 0 1 0⎤00 1 1 1 0 1 0 ⎦ . (13.28)1 0 1 1 0 0 1The dual of the (7, 4) Hamming code H (7,4) is the code shown in table 13.16.00000000010111010110101110101001110101100111000111110100A possibly unexpected property of this pair of codes is that the dual,H ⊥ (7,4) , is contained within the code H (7,4) itself: every word in the dual codeis a codeword of the original (7, 4) Hamming code. This relationship can bewritten using set notation:Table 13.16. The eight codewordsof the dual of the (7, 4) Hammingcode. [Compare with table 1.14,p.9.]H ⊥ (7,4) ⊂ H (7,4). (13.29)The possibility that the set of dual vectors can overlap the set of codewordvectors is counterintuitive if we think of the vectors as real vectors – how cana vector be orthogonal to itself? But when we work in modulo-two arithmetic,many non-zero vectors are indeed orthogonal to themselves!⊲ Exercise 13.7. [1, p.223] Give a simple rule that distinguishes whether a binaryvector is orthogonal to itself, as is each of the three vectors [1 1 1 0 1 0 0],[0 1 1 1 0 1 0], and [1 0 1 1 0 0 1].Some more dualsIn general, if a code has a systematic generator matrix,G = [I K |P T ] , (13.30)where P is a K × M matrix, then its parity-check matrix isH = [P|I M ] . (13.31)